Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h
@ 2021-03-23  9:34 Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
                   ` (52 more replies)
  0 siblings, 53 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:34 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch-set moves all MEM-related functions to mem.c and mem.h. This is done
to encapsulate struct MEM whicl allows us to simplify addition of new field
types.

https://github.com/tarantool/tarantool/issues/5818
https://github.com/tarantool/tarantool/tree/imeevma/gh-5818-encapsulate-mem-type-checking-and-changing

Changes in v4:
  - Patch-set was completely reworked.

Changes in v3:
  - Inlined most of the introduced functions to improve performance.
  - Some other fixes in code to improve performance.

Changes in v2:
  - Squashed almost all patches.
  - Review fixes.

Mergen Imeev (53):
  sql: enchance vdbe_decode_msgpack_into_mem()
  sql: disable unused code in sql/analyze.c
  sql: disable unused code in sql/legacy.c
  sql: remove NULL-termination in OP_ResultRow
  sql: move MEM-related functions to mem.c/mem.h
  sql: remove unused MEM-related functions
  sql: disable unused code in sql/vdbemem.c
  sql: introduce mem_str()
  sql: introduce mem_create()
  sql: introduce mem_destroy()
  sql: introduce mem_is_*() functions()
  sql: introduce mem_copy()
  sql: introduce mem_copy_as_ephemeral()
  sql: rework mem_move()
  sql: rework vdbe_decode_msgpack_into_mem()
  sql: remove sql_column_to_messagepack()
  sql: introduce mem_concat()
  sql: introduce mem_arithmetic()
  sql: introduce mem_compare()
  sql: introduce mem_bitwise()
  sql: introduce mem_bit_not()
  sql: Initialize MEM in sqlVdbeAllocUnpackedRecord()
  sql: introduce mem_set_null()
  sql: introduce mem_set_integer()
  sql: introduce mem_set_unsigned()
  sql: introduce mem_set_boolean()
  sql: refactor mem_set_double()
  sql: refactor mem_set_*_string()
  sql: introduce mem_copy_string()
  sql: introduce mem_set_*_binary()
  sql: introduce mem_copy_binary()
  sql: introduce mem_set_zerobinary()
  sql: introduce mem_append_to_binary()
  sql: introduce mem_set_*_map() and mem_set_*_array()
  sql: introduce mem_set_undefined()
  sql: introduce mem_set_pointer()
  sql: introduce mem_set_frame()
  sql: introduce mem_*_aggregate()
  sql: introduce mem_set_cleared()
  sql: move MEM flags to mem.c
  sql: introduce mem_convert_to_integer()
  sql: introduce mem_convert_to_double()
  sql: introduce mem_convert_to_number()
  sql: introduce mem_convert_to_string()
  sql: introduce mem_explicit_cast()
  sql: introduce mem_implicit_cast()
  sql: introduce mem_get_integer()
  sql: introduce mem_get_unsigned()
  sql: introduce mem_get_double()
  sql: introduce mem_get_boolean()
  sql: introduce mem_get_string0()
  sql: introduce mem_get_binary()
  sql: introduce mem_get_length()

 src/box/CMakeLists.txt  |    1 +
 src/box/execute.c       |  130 +-
 src/box/sql.c           |   96 +-
 src/box/sql.h           |    8 -
 src/box/sql/analyze.c   |   82 +-
 src/box/sql/build.c     |    1 +
 src/box/sql/func.c      |  370 +++--
 src/box/sql/insert.c    |    1 +
 src/box/sql/legacy.c    |    4 +
 src/box/sql/main.c      |    1 +
 src/box/sql/mem.c       | 2994 +++++++++++++++++++++++++++++++++++++++
 src/box/sql/mem.h       |  530 +++++++
 src/box/sql/pragma.c    |    1 +
 src/box/sql/printf.c    |   16 +-
 src/box/sql/select.c    |    3 +-
 src/box/sql/sqlInt.h    |  167 +--
 src/box/sql/trigger.c   |    1 +
 src/box/sql/vdbe.c      | 1220 +++-------------
 src/box/sql/vdbe.h      |    1 -
 src/box/sql/vdbeInt.h   |  314 +---
 src/box/sql/vdbeapi.c   |  438 ++----
 src/box/sql/vdbeaux.c   |  656 +--------
 src/box/sql/vdbemem.c   | 1324 +----------------
 src/box/sql/vdbesort.c  |    9 +-
 src/box/sql/vdbetrace.c |   34 +-
 src/box/sql/where.c     |    1 +
 src/box/sql/whereexpr.c |    7 +-
 27 files changed, 4260 insertions(+), 4150 deletions(-)
 create mode 100644 src/box/sql/mem.c
 create mode 100644 src/box/sql/mem.h

-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:34 ` Mergen Imeev via Tarantool-patches
  2021-03-29 22:57   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 02/53] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches
                   ` (51 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:34 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
properly initialized in case msgpack contains MP_EXT, MP_MAP, or
MP_ARRAY fields. Also, it doesn't set field_type.

Needed for #5818
---
 src/box/sql/vdbe.c    | 22 ---------------
 src/box/sql/vdbeaux.c | 64 ++++++++++++++++++++++++++++++++-----------
 2 files changed, 48 insertions(+), 38 deletions(-)

diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 3b3b1f01d..9a4f38bb9 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -846,16 +846,6 @@ vdbe_field_ref_fetch_data(struct vdbe_field_ref *field_ref, uint32_t fieldno)
 	return field_begin;
 }
 
-static inline enum field_type
-vdbe_field_ref_fetch_type(struct vdbe_field_ref *field_ref, uint32_t fieldno)
-{
-	const struct tuple_field *tf =
-		vdbe_field_ref_fetch_field(field_ref, fieldno);
-	if (tf == NULL || tf->type == FIELD_TYPE_ANY)
-		return field_type_MAX;
-	return tf->type;
-}
-
 /**
  * Fetch field by fieldno using vdbe_field_ref and store result
  * in dest_mem.
@@ -879,17 +869,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
 	if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0)
 		return -1;
 
-	/*
-	 * MsgPack map, array or extension (unsupported in sql).
-	 * Wrap it in a blob verbatim.
-	 */
-	if (dest_mem->flags == 0) {
-		dest_mem->z = (char *) data;
-		dest_mem->n = vdbe_field_ref_fetch_data(field_ref,
-							fieldno + 1) - data;
-		dest_mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
-		dest_mem->subtype = SQL_SUBTYPE_MSGPACK;
-	}
 	/*
 	 * Add 0 termination (at most for strings)
 	 * Not sure why do we check MEM_Ephem
@@ -909,7 +888,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
 		dest_mem->flags |= MEM_Term;
 	}
 	UPDATE_MAX_BLOBSIZE(dest_mem);
-	dest_mem->field_type = vdbe_field_ref_fetch_type(field_ref, fieldno);
 	return 0;
 }
 
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 91b64316e..90c4b73c0 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2793,38 +2793,62 @@ vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
 {
 	const char *start_buf = buf;
 	switch (mp_typeof(*buf)) {
-	case MP_ARRAY:
-	case MP_MAP:
-	case MP_EXT:
-	default: {
-		mem->flags = 0;
+	case MP_ARRAY: {
+		mem->z = (char *)buf;
+		mp_next(&buf);
+		mem->n = buf - mem->z;
+		mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+		mem->subtype = SQL_SUBTYPE_MSGPACK;
+		mem->field_type = FIELD_TYPE_ARRAY;
+		break;
+	}
+	case MP_MAP: {
+		mem->z = (char *)buf;
+		mp_next(&buf);
+		mem->n = buf - mem->z;
+		mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+		mem->subtype = SQL_SUBTYPE_MSGPACK;
+		mem->field_type = FIELD_TYPE_MAP;
+		break;
+	}
+	case MP_EXT: {
+		mem->z = (char *)buf;
+		mp_next(&buf);
+		mem->n = buf - mem->z;
+		mem->flags = MEM_Blob | MEM_Ephem;
+		mem->field_type = FIELD_TYPE_VARBINARY;
 		break;
 	}
 	case MP_NIL: {
 		mp_decode_nil(&buf);
 		mem->flags = MEM_Null;
+		mem->field_type = field_type_MAX;
 		break;
 	}
 	case MP_BOOL: {
 		mem->u.b = mp_decode_bool(&buf);
 		mem->flags = MEM_Bool;
+		mem->field_type = FIELD_TYPE_BOOLEAN;
 		break;
 	}
 	case MP_UINT: {
 		uint64_t v = mp_decode_uint(&buf);
 		mem->u.u = v;
 		mem->flags = MEM_UInt;
+		mem->field_type = FIELD_TYPE_INTEGER;
 		break;
 	}
 	case MP_INT: {
 		mem->u.i = mp_decode_int(&buf);
 		mem->flags = MEM_Int;
+		mem->field_type = FIELD_TYPE_INTEGER;
 		break;
 	}
 	case MP_STR: {
 		/* XXX u32->int */
 		mem->n = (int) mp_decode_strl(&buf);
 		mem->flags = MEM_Str | MEM_Ephem;
+		mem->field_type = FIELD_TYPE_STRING;
 install_blob:
 		mem->z = (char *)buf;
 		buf += mem->n;
@@ -2834,18 +2858,33 @@ install_blob:
 		/* XXX u32->int */
 		mem->n = (int) mp_decode_binl(&buf);
 		mem->flags = MEM_Blob | MEM_Ephem;
+		mem->field_type = FIELD_TYPE_VARBINARY;
 		goto install_blob;
 	}
 	case MP_FLOAT: {
 		mem->u.r = mp_decode_float(&buf);
-		mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
+		if (sqlIsNaN(mem->u.r)) {
+			mem->flags = MEM_Null;
+			mem->field_type = field_type_MAX;
+		} else {
+			mem->flags = MEM_Real;
+			mem->field_type = FIELD_TYPE_DOUBLE;
+		}
 		break;
 	}
 	case MP_DOUBLE: {
 		mem->u.r = mp_decode_double(&buf);
-		mem->flags = sqlIsNaN(mem->u.r) ? MEM_Null : MEM_Real;
+		if (sqlIsNaN(mem->u.r)) {
+			mem->flags = MEM_Null;
+			mem->field_type = field_type_MAX;
+		} else {
+			mem->flags = MEM_Real;
+			mem->field_type = FIELD_TYPE_DOUBLE;
+		}
 		break;
 	}
+	default:
+		unreachable();
 	}
 	*len = (uint32_t)(buf - start_buf);
 	return 0;
@@ -2868,15 +2907,8 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def,	/* Information about the rec
 		pMem->z = 0;
 		uint32_t sz = 0;
 		vdbe_decode_msgpack_into_mem(zParse, pMem, &sz);
-		if (sz == 0) {
-			/* MsgPack array, map or ext. Treat as blob. */
-			pMem->z = (char *)zParse;
-			mp_next(&zParse);
-			pMem->n = zParse - pMem->z;
-			pMem->flags = MEM_Blob | MEM_Ephem;
-		} else {
-			zParse += sz;
-		}
+		assert(sz != 0);
+		zParse += sz;
 		pMem++;
 	}
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 02/53] sql: disable unused code in sql/analyze.c
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:34 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 03/53] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches
                   ` (50 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:34 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch disables unused code in sql/analyze.c. It will simplify
refactoring.

Needed for #5818
---
 src/box/sql.c         | 95 +++++++++++++++++++++++++++++++++++--------
 src/box/sql.h         |  8 ----
 src/box/sql/analyze.c | 81 ++----------------------------------
 src/box/sql/vdbe.c    |  3 ++
 4 files changed, 84 insertions(+), 103 deletions(-)

diff --git a/src/box/sql.c b/src/box/sql.c
index 3b53abcdb..aa91f003f 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -78,24 +78,6 @@ sql_init(void)
 	assert(db != NULL);
 }
 
-void
-sql_load_schema(void)
-{
-	assert(db->init.busy == 0);
-	struct space *stat = space_by_name("_sql_stat1");
-	assert(stat != NULL);
-	if (stat->def->field_count == 0)
-		return;
-	db->init.busy = 1;
-	if (sql_analysis_load(db) != 0) {
-		if(!diag_is_empty(&fiber()->diag)) {
-			diag_log();
-		}
-		panic("failed to initialize SQL subsystem");
-	}
-	db->init.busy = 0;
-}
-
 sql *
 sql_get(void)
 {
@@ -1317,3 +1299,80 @@ vdbe_field_ref_prepare_tuple(struct vdbe_field_ref *field_ref,
 	vdbe_field_ref_create(field_ref, tuple, tuple_data(tuple),
 			      tuple->bsize);
 }
+
+ssize_t
+sql_index_tuple_size(struct space *space, struct index *idx)
+{
+	assert(space != NULL);
+	assert(idx != NULL);
+	assert(idx->def->space_id == space->def->id);
+	ssize_t tuple_count = index_size(idx);
+	ssize_t space_size = space_bsize(space);
+	ssize_t avg_tuple_size = tuple_count != 0 ?
+				 (space_size / tuple_count) : 0;
+	return avg_tuple_size;
+}
+
+/**
+ * default_tuple_est[] array contains default information
+ * which is used when we don't have real space, e.g. temporary
+ * objects representing result set of nested SELECT or VIEW.
+ *
+ * First number is supposed to contain the number of elements
+ * in the index. Since we do not know, guess 1 million.
+ * Second one is an estimate of the number of rows in the
+ * table that match any particular value of the first column of
+ * the index. Third one is an estimate of the number of
+ * rows that match any particular combination of the first 2
+ * columns of the index. And so on. It must always be true:
+ *
+ *           default_tuple_est[N] <= default_tuple_est[N-1]
+ *           default_tuple_est[N] >= 1
+ *
+ * Apart from that, we have little to go on besides intuition
+ * as to how default values should be initialized. The numbers
+ * generated here are based on typical values found in actual
+ * indices.
+ */
+const log_est_t default_tuple_est[] = {DEFAULT_TUPLE_LOG_COUNT,
+/**                  [10*log_{2}(x)]:  10, 9,  8,  7,  6,  5 */
+				       33, 32, 30, 28, 26, 23};
+
+LogEst
+sql_space_tuple_log_count(struct space *space)
+{
+	if (space == NULL || space->index_map == NULL)
+		return 0;
+
+	struct index *pk = space_index(space, 0);
+	assert(sqlLogEst(DEFAULT_TUPLE_COUNT) == DEFAULT_TUPLE_LOG_COUNT);
+	/* If space represents VIEW, return default number. */
+	if (pk == NULL)
+		return DEFAULT_TUPLE_LOG_COUNT;
+	return sqlLogEst(pk->vtab->size(pk));
+}
+
+log_est_t
+index_field_tuple_est(const struct index_def *idx_def, uint32_t field)
+{
+	assert(idx_def != NULL);
+	struct space *space = space_by_id(idx_def->space_id);
+	if (space == NULL || strcmp(idx_def->name, "fake_autoindex") == 0)
+		return idx_def->opts.stat->tuple_log_est[field];
+	assert(field <= idx_def->key_def->part_count);
+	/* Statistics is held only in real indexes. */
+	struct index *tnt_idx = space_index(space, idx_def->iid);
+	assert(tnt_idx != NULL);
+	if (tnt_idx->def->opts.stat == NULL) {
+		/*
+		 * Last number for unique index is always 0:
+		 * only one tuple exists with given full key
+		 * in unique index and log(1) == 0.
+		 */
+		if (field == idx_def->key_def->part_count &&
+		    idx_def->opts.is_unique)
+			return 0;
+		return default_tuple_est[field + 1 >= 6 ? 6 : field];
+	}
+	return tnt_idx->def->opts.stat->tuple_log_est[field];
+}
diff --git a/src/box/sql.h b/src/box/sql.h
index f56f7a1f1..4c364306c 100644
--- a/src/box/sql.h
+++ b/src/box/sql.h
@@ -41,14 +41,6 @@ extern "C" {
 void
 sql_init(void);
 
-/**
- * Initialize SQL statistic system.
- *
- * Currently unused.
- */
-void
-sql_load_schema(void);
-
 /**
  * struct sql *
  * sql_get();
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index f74f9b358..a015f70cb 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -116,6 +116,8 @@
 #include "tarantoolInt.h"
 #include "vdbeInt.h"
 
+#if 0
+
 /**
  * This routine generates code that opens the sql_stat1/4 tables.
  * If the sql_statN tables do not previously exist, they are
@@ -1114,19 +1116,6 @@ sqlAnalyze(Parse * pParse, Token * pName)
 		sqlVdbeAddOp0(v, OP_Expire);
 }
 
-ssize_t
-sql_index_tuple_size(struct space *space, struct index *idx)
-{
-	assert(space != NULL);
-	assert(idx != NULL);
-	assert(idx->def->space_id == space->def->id);
-	ssize_t tuple_count = index_size(idx);
-	ssize_t space_size = space_bsize(space);
-	ssize_t avg_tuple_size = tuple_count != 0 ?
-				 (space_size / tuple_count) : 0;
-	return avg_tuple_size;
-}
-
 /**
  * Used to pass information from the analyzer reader through
  * to the callback routine.
@@ -1538,70 +1527,6 @@ load_stat_to_index(const char *sql_select_load, struct index_stat **stats)
 	return 0;
 }
 
-/**
- * default_tuple_est[] array contains default information
- * which is used when we don't have real space, e.g. temporary
- * objects representing result set of nested SELECT or VIEW.
- *
- * First number is supposed to contain the number of elements
- * in the index. Since we do not know, guess 1 million.
- * Second one is an estimate of the number of rows in the
- * table that match any particular value of the first column of
- * the index. Third one is an estimate of the number of
- * rows that match any particular combination of the first 2
- * columns of the index. And so on. It must always be true:
- *
- *           default_tuple_est[N] <= default_tuple_est[N-1]
- *           default_tuple_est[N] >= 1
- *
- * Apart from that, we have little to go on besides intuition
- * as to how default values should be initialized. The numbers
- * generated here are based on typical values found in actual
- * indices.
- */
-const log_est_t default_tuple_est[] = {DEFAULT_TUPLE_LOG_COUNT,
-/**                  [10*log_{2}(x)]:  10, 9,  8,  7,  6,  5 */
-				       33, 32, 30, 28, 26, 23};
-
-LogEst
-sql_space_tuple_log_count(struct space *space)
-{
-	if (space == NULL || space->index_map == NULL)
-		return 0;
-
-	struct index *pk = space_index(space, 0);
-	assert(sqlLogEst(DEFAULT_TUPLE_COUNT) == DEFAULT_TUPLE_LOG_COUNT);
-	/* If space represents VIEW, return default number. */
-	if (pk == NULL)
-		return DEFAULT_TUPLE_LOG_COUNT;
-	return sqlLogEst(pk->vtab->size(pk));
-}
-
-log_est_t
-index_field_tuple_est(const struct index_def *idx_def, uint32_t field)
-{
-	assert(idx_def != NULL);
-	struct space *space = space_by_id(idx_def->space_id);
-	if (space == NULL || strcmp(idx_def->name, "fake_autoindex") == 0)
-		return idx_def->opts.stat->tuple_log_est[field];
-	assert(field <= idx_def->key_def->part_count);
-	/* Statistics is held only in real indexes. */
-	struct index *tnt_idx = space_index(space, idx_def->iid);
-	assert(tnt_idx != NULL);
-	if (tnt_idx->def->opts.stat == NULL) {
-		/*
-		 * Last number for unique index is always 0:
-		 * only one tuple exists with given full key
-		 * in unique index and log(1) == 0.
-		 */
-		if (field == idx_def->key_def->part_count &&
-		    idx_def->opts.is_unique)
-			return 0;
-		return default_tuple_est[field + 1 >= 6 ? 6 : field];
-	}
-	return tnt_idx->def->opts.stat->tuple_log_est[field];
-}
-
 /**
  * This function performs copy of statistics.
  * In contrast to index_stat_dup(), there is no assumption
@@ -1744,3 +1669,5 @@ fail:
 	box_txn_rollback();
 	return -1;
 }
+
+#endif
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 9a4f38bb9..4c1cd582b 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4825,8 +4825,11 @@ case OP_RenameTable: {
  */
 case OP_LoadAnalysis: {
 	assert(pOp->p1==0 );
+	/* TODO: Enable analysis. */
+	/*
 	if (sql_analysis_load(db) != 0)
 		goto abort_due_to_error;
+	*/
 	break;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 03/53] sql: disable unused code in sql/legacy.c
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 02/53] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:34 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 04/53] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches
                   ` (49 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:34 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch disables unused code in sql/legacy.c. It will simplify
refactoring.

Needed for #5818
---
 src/box/sql/legacy.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/box/sql/legacy.c b/src/box/sql/legacy.c
index 072954357..b1781b427 100644
--- a/src/box/sql/legacy.c
+++ b/src/box/sql/legacy.c
@@ -40,6 +40,8 @@
 #include "box/execute.h"
 #include "box/session.h"
 
+#if 0
+
 /*
  * Execute SQL code.  Return one of the SQL_ success/failure
  * codes.
@@ -162,3 +164,5 @@ sql_exec(sql * db,	/* The database on which the SQL executes */
 	assert(rc == 0);
 	return rc;
 }
+
+#endif
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 04/53] sql: remove NULL-termination in OP_ResultRow
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (2 preceding siblings ...)
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 03/53] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:34 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 05/53] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (48 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:34 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

There is no need for NULL-termination, since MEMs converted to msgpack
and msgpack do not require NULL-terminated strings.

Needed for #5818
---
 src/box/sql/vdbe.c | 19 +++++--------------
 1 file changed, 5 insertions(+), 14 deletions(-)

diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4c1cd582b..18806b93f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1505,9 +1505,6 @@ case OP_SCopy: {            /* out2 */
  * the result row.
  */
 case OP_ResultRow: {
-	Mem *pMem;
-	int i;
-
 	assert(p->nResColumn==pOp->p2);
 	assert(pOp->p1>0);
 	assert(pOp->p1+pOp->p2<=(p->nMem+1 - p->nCursor)+1);
@@ -1516,20 +1513,14 @@ case OP_ResultRow: {
 	/* Invalidate all ephemeral cursor row caches */
 	p->cacheCtr = (p->cacheCtr + 2)|1;
 
-	/* Make sure the results of the current row are \000 terminated
-	 * and have an assigned type.  The results are de-ephemeralized as
-	 * a side effect.
-	 */
-	pMem = p->pResultSet = &aMem[pOp->p1];
-	for(i=0; i<pOp->p2; i++) {
+	p->pResultSet = &aMem[pOp->p1];
+#ifdef SQL_DEBUG
+	struct Mem *pMem = p->pResultSet;
+	for(int i = 0; i < pOp->p2; i++) {
 		assert(memIsValid(&pMem[i]));
-		Deephemeralize(&pMem[i]);
-		assert((pMem[i].flags & MEM_Ephem)==0
-		       || (pMem[i].flags & (MEM_Str|MEM_Blob))==0);
-		sqlVdbeMemNulTerminate(&pMem[i]);
 		REGISTER_TRACE(p, pOp->p1+i, &pMem[i]);
 	}
-	if (db->mallocFailed) goto no_mem;
+#endif
 
 	if (db->mTrace & SQL_TRACE_ROW) {
 		db->xTrace(SQL_TRACE_ROW, db->pTraceArg, p, 0);
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 05/53] sql: move MEM-related functions to mem.c/mem.h
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (3 preceding siblings ...)
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 04/53] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:34 ` Mergen Imeev via Tarantool-patches
  2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 06/53] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches
                   ` (47 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:34 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch moves all MEM-related functions in mem.c/mem.h.

Part of #5818
---
 src/box/CMakeLists.txt  |    1 +
 src/box/sql.c           |    1 +
 src/box/sql/analyze.c   |    1 +
 src/box/sql/build.c     |    1 +
 src/box/sql/func.c      |    1 +
 src/box/sql/insert.c    |    1 +
 src/box/sql/main.c      |    1 +
 src/box/sql/mem.c       | 2376 +++++++++++++++++++++++++++++++++++++++
 src/box/sql/mem.h       |  502 +++++++++
 src/box/sql/pragma.c    |    1 +
 src/box/sql/printf.c    |    1 +
 src/box/sql/select.c    |    1 +
 src/box/sql/sqlInt.h    |   50 -
 src/box/sql/trigger.c   |    1 +
 src/box/sql/vdbe.c      |  471 +-------
 src/box/sql/vdbe.h      |    1 -
 src/box/sql/vdbeInt.h   |  299 -----
 src/box/sql/vdbeapi.c   |  137 +--
 src/box/sql/vdbeaux.c   |  567 +---------
 src/box/sql/vdbemem.c   | 1241 +-------------------
 src/box/sql/vdbesort.c  |    1 +
 src/box/sql/vdbetrace.c |    1 +
 src/box/sql/where.c     |    1 +
 src/box/sql/whereexpr.c |    1 +
 24 files changed, 2897 insertions(+), 2762 deletions(-)
 create mode 100644 src/box/sql/mem.c
 create mode 100644 src/box/sql/mem.h

diff --git a/src/box/CMakeLists.txt b/src/box/CMakeLists.txt
index 19203f770..41c32468f 100644
--- a/src/box/CMakeLists.txt
+++ b/src/box/CMakeLists.txt
@@ -42,6 +42,7 @@ set(sql_sources
     sql/legacy.c
     sql/main.c
     sql/malloc.c
+    sql/mem.c
     sql/os.c
     sql/os_unix.c
     sql/parse_def.c
diff --git a/src/box/sql.c b/src/box/sql.c
index aa91f003f..790ca7f70 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -34,6 +34,7 @@
 #include "sql.h"
 #include "sql/sqlInt.h"
 #include "sql/tarantoolInt.h"
+#include "sql/mem.h"
 #include "sql/vdbeInt.h"
 
 #include "index.h"
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index a015f70cb..5a95caa39 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -114,6 +114,7 @@
 
 #include "sqlInt.h"
 #include "tarantoolInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 
 #if 0
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 521cc2622..6470e0300 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -45,6 +45,7 @@
  */
 #include <ctype.h>
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "tarantoolInt.h"
 #include "box/ck_constraint.h"
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index f15d27051..074d41260 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -35,6 +35,7 @@
  * time functions, are implemented separately.)
  */
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "version.h"
 #include "coll/coll.h"
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index 588e142d2..02e9f9673 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -35,6 +35,7 @@
  */
 #include "sqlInt.h"
 #include "tarantoolInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "box/ck_constraint.h"
 #include "bit/bit.h"
diff --git a/src/box/sql/main.c b/src/box/sql/main.c
index 0b20f2132..b0d32ae32 100644
--- a/src/box/sql/main.c
+++ b/src/box/sql/main.c
@@ -36,6 +36,7 @@
  * accessed by users of the library.
  */
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "version.h"
 #include "box/session.h"
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
new file mode 100644
index 000000000..62338e1db
--- /dev/null
+++ b/src/box/sql/mem.c
@@ -0,0 +1,2376 @@
+/*
+ * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ *    copyright notice, this list of conditions and the
+ *    following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ *    copyright notice, this list of conditions and the following
+ *    disclaimer in the documentation and/or other materials
+ *    provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include "sqlInt.h"
+#include "mem.h"
+#include "vdbeInt.h"
+#include "coll/coll.h"
+#include "tarantoolInt.h"
+#include "box/schema.h"
+#include "box/tuple.h"
+#include "mpstream/mpstream.h"
+
+static inline bool
+mem_has_msgpack_subtype(struct Mem *mem)
+{
+	return (mem->flags & MEM_Subtype) != 0 &&
+	       mem->subtype == SQL_SUBTYPE_MSGPACK;
+}
+
+/*
+ * The pVal argument is known to be a value other than NULL.
+ * Convert it into a string with encoding enc and return a pointer
+ * to a zero-terminated version of that string.
+ */
+static SQL_NOINLINE const void *
+valueToText(sql_value * pVal)
+{
+	assert(pVal != 0);
+	assert((pVal->flags & (MEM_Null)) == 0);
+	if ((pVal->flags & (MEM_Blob | MEM_Str)) &&
+	    !mem_has_msgpack_subtype(pVal)) {
+		if (ExpandBlob(pVal))
+			return 0;
+		pVal->flags |= MEM_Str;
+		sqlVdbeMemNulTerminate(pVal);	/* IMP: R-31275-44060 */
+	} else {
+		sqlVdbeMemStringify(pVal);
+		assert(0 == (1 & SQL_PTR_TO_INT(pVal->z)));
+	}
+	return pVal->z;
+}
+
+/**
+ * According to ANSI SQL string value can be converted to boolean
+ * type if string consists of literal "true" or "false" and
+ * number of leading and trailing spaces.
+ *
+ * For instance, "   tRuE  " can be successfully converted to
+ * boolean value true.
+ *
+ * @param str String to be converted to boolean. Assumed to be
+ *        null terminated.
+ * @param[out] result Resulting value of cast.
+ * @retval 0 If string satisfies conditions above.
+ * @retval -1 Otherwise.
+ */
+static int
+str_cast_to_boolean(const char *str, bool *result)
+{
+	assert(str != NULL);
+	for (; *str == ' '; str++);
+	if (strncasecmp(str, SQL_TOKEN_TRUE, strlen(SQL_TOKEN_TRUE)) == 0) {
+		*result = true;
+		str += 4;
+	} else if (strncasecmp(str, SQL_TOKEN_FALSE,
+			       strlen(SQL_TOKEN_FALSE)) == 0) {
+		*result = false;
+		str += 5;
+	} else {
+		return -1;
+	}
+	for (; *str != '\0'; ++str) {
+		if (*str != ' ')
+			return -1;
+	}
+	return 0;
+}
+
+/*
+ * Convert a 64-bit IEEE double into a 64-bit signed integer.
+ * If the double is out of range of a 64-bit signed integer then
+ * return the closest available 64-bit signed integer.
+ */
+static int
+doubleToInt64(double r, int64_t *i)
+{
+	/*
+	 * Many compilers we encounter do not define constants for the
+	 * minimum and maximum 64-bit integers, or they define them
+	 * inconsistently.  And many do not understand the "LL" notation.
+	 * So we define our own static constants here using nothing
+	 * larger than a 32-bit integer constant.
+	 */
+	static const int64_t maxInt = LARGEST_INT64;
+	static const int64_t minInt = SMALLEST_INT64;
+	if (r <= (double)minInt) {
+		*i = minInt;
+		return -1;
+	} else if (r >= (double)maxInt) {
+		*i = maxInt;
+		return -1;
+	} else {
+		*i = (int64_t) r;
+		return *i != r;
+	}
+}
+
+/*
+ * It is already known that pMem contains an unterminated string.
+ * Add the zero terminator.
+ */
+static SQL_NOINLINE int
+vdbeMemAddTerminator(Mem * pMem)
+{
+	if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
+		return -1;
+	}
+	pMem->z[pMem->n] = 0;
+	pMem->z[pMem->n + 1] = 0;
+	pMem->flags |= MEM_Term;
+	return 0;
+}
+
+/*
+ * If the memory cell contains a value that must be freed by
+ * invoking the external callback in Mem.xDel, then this routine
+ * will free that value.  It also sets Mem.flags to MEM_Null.
+ *
+ * This is a helper routine for sqlVdbeMemSetNull() and
+ * for sqlVdbeMemRelease().  Use those other routines as the
+ * entry point for releasing Mem resources.
+ */
+static SQL_NOINLINE void
+vdbeMemClearExternAndSetNull(Mem * p)
+{
+	assert(VdbeMemDynamic(p));
+	if (p->flags & MEM_Agg) {
+		sql_vdbemem_finalize(p, p->u.func);
+		assert((p->flags & MEM_Agg) == 0);
+		testcase(p->flags & MEM_Dyn);
+	}
+	if (p->flags & MEM_Dyn) {
+		assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
+		p->xDel((void *)p->z);
+	} else if (p->flags & MEM_Frame) {
+		VdbeFrame *pFrame = p->u.pFrame;
+		pFrame->pParent = pFrame->v->pDelFrame;
+		pFrame->v->pDelFrame = pFrame;
+	}
+	p->flags = MEM_Null;
+}
+
+/*
+ * Release memory held by the Mem p, both external memory cleared
+ * by p->xDel and memory in p->zMalloc.
+ *
+ * This is a helper routine invoked by sqlVdbeMemRelease() in
+ * the unusual case where there really is memory in p that needs
+ * to be freed.
+ */
+static SQL_NOINLINE void
+vdbeMemClear(Mem * p)
+{
+	if (VdbeMemDynamic(p)) {
+		vdbeMemClearExternAndSetNull(p);
+	}
+	if (p->szMalloc) {
+		sqlDbFree(p->db, p->zMalloc);
+		p->szMalloc = 0;
+	}
+	p->z = 0;
+}
+
+/*
+ * Make an shallow copy of pFrom into pTo.  Prior contents of
+ * pTo are freed.  The pFrom->z field is not duplicated.  If
+ * pFrom->z is used, then pTo->z points to the same thing as pFrom->z
+ * and flags gets srcType (either MEM_Ephem or MEM_Static).
+ */
+static SQL_NOINLINE void
+vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
+{
+	vdbeMemClearExternAndSetNull(pTo);
+	assert(!VdbeMemDynamic(pTo));
+	sqlVdbeMemShallowCopy(pTo, pFrom, eType);
+}
+
+/*
+ * Both *pMem1 and *pMem2 contain string values. Compare the two values
+ * using the collation sequence pColl. As usual, return a negative , zero
+ * or positive value if *pMem1 is less than, equal to or greater than
+ * *pMem2, respectively. Similar in spirit to "rc = (*pMem1) - (*pMem2);".
+ *
+ * Strungs assume to be UTF-8 encoded
+ */
+static int
+vdbeCompareMemString(const Mem * pMem1, const Mem * pMem2,
+		     const struct coll * pColl)
+{
+	return pColl->cmp(pMem1->z, (size_t)pMem1->n,
+			      pMem2->z, (size_t)pMem2->n, pColl);
+}
+
+/*
+ * The input pBlob is guaranteed to be a Blob that is not marked
+ * with MEM_Zero.  Return true if it could be a zero-blob.
+ */
+static int
+isAllZero(const char *z, int n)
+{
+	int i;
+	for (i = 0; i < n; i++) {
+		if (z[i])
+			return 0;
+	}
+	return 1;
+}
+
+char *
+mem_type_to_str(const struct Mem *p)
+{
+	assert(p != NULL);
+	switch (p->flags & MEM_PURE_TYPE_MASK) {
+	case MEM_Null:
+		return "NULL";
+	case MEM_Str:
+		return "text";
+	case MEM_Int:
+		return "integer";
+	case MEM_UInt:
+		return "unsigned";
+	case MEM_Real:
+		return "real";
+	case MEM_Blob:
+		return "varbinary";
+	case MEM_Bool:
+		return "boolean";
+	default:
+		unreachable();
+	}
+}
+
+enum mp_type
+mem_mp_type(struct Mem *mem)
+{
+	switch (mem->flags & MEM_PURE_TYPE_MASK) {
+	case MEM_Int:
+		return MP_INT;
+	case MEM_UInt:
+		return MP_UINT;
+	case MEM_Real:
+		return MP_DOUBLE;
+	case MEM_Str:
+		return MP_STR;
+	case MEM_Blob:
+		if ((mem->flags & MEM_Subtype) == 0 ||
+		     mem->subtype != SQL_SUBTYPE_MSGPACK)
+			return MP_BIN;
+		assert(mp_typeof(*mem->z) == MP_MAP ||
+		       mp_typeof(*mem->z) == MP_ARRAY);
+		return mp_typeof(*mem->z);
+	case MEM_Bool:
+		return MP_BOOL;
+	case MEM_Null:
+		return MP_NIL;
+	default: unreachable();
+	}
+}
+
+/* EVIDENCE-OF: R-12793-43283 Every value in sql has one of five
+ * fundamental datatypes: 64-bit signed integer 64-bit IEEE floating
+ * point number string BLOB NULL
+ */
+enum mp_type
+sql_value_type(sql_value *pVal)
+{
+	struct Mem *mem = (struct Mem *) pVal;
+	return mem_mp_type(mem);
+}
+
+
+/*
+ * pMem currently only holds a string type (or maybe a BLOB that we can
+ * interpret as a string if we want to).  Compute its corresponding
+ * numeric type, if has one.  Set the pMem->u.r and pMem->u.i fields
+ * accordingly.
+ */
+static u16 SQL_NOINLINE
+computeNumericType(Mem *pMem)
+{
+	assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
+	assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
+	if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
+		return 0;
+	bool is_neg;
+	if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
+		return is_neg ? MEM_Int : MEM_UInt;
+	return MEM_Real;
+}
+
+/*
+ * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
+ * none.
+ *
+ * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
+ * But it does set pMem->u.r and pMem->u.i appropriately.
+ */
+u16
+numericType(Mem *pMem)
+{
+	if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+		return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
+	if (pMem->flags & (MEM_Str|MEM_Blob)) {
+		return computeNumericType(pMem);
+	}
+	return 0;
+}
+
+/*
+ * The sqlValueBytes() routine returns the number of bytes in the
+ * sql_value object assuming that it uses the encoding "enc".
+ * The valueBytes() routine is a helper function.
+ */
+static SQL_NOINLINE int
+valueBytes(sql_value * pVal)
+{
+	return valueToText(pVal) != 0 ? pVal->n : 0;
+}
+
+int
+sqlValueBytes(sql_value * pVal)
+{
+	Mem *p = (Mem *) pVal;
+	assert((p->flags & MEM_Null) == 0
+	       || (p->flags & (MEM_Str | MEM_Blob)) == 0);
+	if ((p->flags & MEM_Str) != 0) {
+		return p->n;
+	}
+	if ((p->flags & MEM_Blob) != 0) {
+		if (p->flags & MEM_Zero) {
+			return p->n + p->u.nZero;
+		} else {
+			return p->n;
+		}
+	}
+	if (p->flags & MEM_Null)
+		return 0;
+	return valueBytes(pVal);
+}
+
+
+#ifdef SQL_DEBUG
+/*
+ * This routine prepares a memory cell for modification by breaking
+ * its link to a shallow copy and by marking any current shallow
+ * copies of this cell as invalid.
+ *
+ * This is used for testing and debugging only - to make sure shallow
+ * copies are not misused.
+ */
+void
+sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
+{
+	int i;
+	Mem *pX;
+	for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
+		if (pX->pScopyFrom == pMem) {
+			pX->flags |= MEM_Undefined;
+			pX->pScopyFrom = 0;
+		}
+	}
+	pMem->pScopyFrom = 0;
+}
+
+/*
+ * Check invariants on a Mem object.
+ *
+ * This routine is intended for use inside of assert() statements, like
+ * this:    assert( sqlVdbeCheckMemInvariants(pMem) );
+ */
+int
+sqlVdbeCheckMemInvariants(Mem * p)
+{
+	/* If MEM_Dyn is set then Mem.xDel!=0.
+	 * Mem.xDel is might not be initialized if MEM_Dyn is clear.
+	 */
+	assert((p->flags & MEM_Dyn) == 0 || p->xDel != 0);
+
+	/* MEM_Dyn may only be set if Mem.szMalloc==0.  In this way we
+	 * ensure that if Mem.szMalloc>0 then it is safe to do
+	 * Mem.z = Mem.zMalloc without having to check Mem.flags&MEM_Dyn.
+	 * That saves a few cycles in inner loops.
+	 */
+	assert((p->flags & MEM_Dyn) == 0 || p->szMalloc == 0);
+
+	/* Cannot be both MEM_Int and MEM_Real at the same time */
+	assert((p->flags & (MEM_Int | MEM_Real)) != (MEM_Int | MEM_Real));
+	/* Can't be both UInt and Int at the same time.  */
+	assert((p->flags & (MEM_Int | MEM_UInt)) != (MEM_Int | MEM_UInt));
+
+	/* The szMalloc field holds the correct memory allocation size */
+	assert(p->szMalloc == 0 ||
+	       p->szMalloc == sqlDbMallocSize(p->db, p->zMalloc));
+
+	/* If p holds a string or blob, the Mem.z must point to exactly
+	 * one of the following:
+	 *
+	 *   (1) Memory in Mem.zMalloc and managed by the Mem object
+	 *   (2) Memory to be freed using Mem.xDel
+	 *   (3) An ephemeral string or blob
+	 *   (4) A static string or blob
+	 */
+	if ((p->flags & (MEM_Str | MEM_Blob)) && p->n > 0) {
+		assert(((p->szMalloc > 0 && p->z == p->zMalloc) ? 1 : 0) +
+		       ((p->flags & MEM_Dyn) != 0 ? 1 : 0) +
+		       ((p->flags & MEM_Ephem) != 0 ? 1 : 0) +
+		       ((p->flags & MEM_Static) != 0 ? 1 : 0) == 1);
+	}
+	return 1;
+}
+
+/*
+ * Print the SQL that was used to generate a VDBE program.
+ */
+void
+sqlVdbePrintSql(Vdbe * p)
+{
+	const char *z = 0;
+	if (p->zSql) {
+		z = p->zSql;
+	} else if (p->nOp >= 1) {
+		const VdbeOp *pOp = &p->aOp[0];
+		if (pOp->opcode == OP_Init && pOp->p4.z != 0) {
+			z = pOp->p4.z;
+			while (sqlIsspace(*z))
+				z++;
+		}
+	}
+	if (z)
+		printf("SQL: [%s]\n", z);
+}
+
+/*
+ * Write a nice string representation of the contents of cell pMem
+ * into buffer zBuf, length nBuf.
+ */
+void
+sqlVdbeMemPrettyPrint(Mem *pMem, char *zBuf)
+{
+	char *zCsr = zBuf;
+	int f = pMem->flags;
+
+	if (f&MEM_Blob) {
+		int i;
+		char c;
+		if (f & MEM_Dyn) {
+			c = 'z';
+			assert((f & (MEM_Static|MEM_Ephem))==0);
+		} else if (f & MEM_Static) {
+			c = 't';
+			assert((f & (MEM_Dyn|MEM_Ephem))==0);
+		} else if (f & MEM_Ephem) {
+			c = 'e';
+			assert((f & (MEM_Static|MEM_Dyn))==0);
+		} else {
+			c = 's';
+		}
+
+		sql_snprintf(100, zCsr, "%c", c);
+		zCsr += sqlStrlen30(zCsr);
+		sql_snprintf(100, zCsr, "%d[", pMem->n);
+		zCsr += sqlStrlen30(zCsr);
+		for(i=0; i<16 && i<pMem->n; i++) {
+			sql_snprintf(100, zCsr, "%02X", ((int)pMem->z[i] & 0xFF));
+			zCsr += sqlStrlen30(zCsr);
+		}
+		for(i=0; i<16 && i<pMem->n; i++) {
+			char z = pMem->z[i];
+			if (z<32 || z>126) *zCsr++ = '.';
+			else *zCsr++ = z;
+		}
+		sql_snprintf(100, zCsr, "]%s", "(8)");
+		zCsr += sqlStrlen30(zCsr);
+		if (f & MEM_Zero) {
+			sql_snprintf(100, zCsr,"+%dz",pMem->u.nZero);
+			zCsr += sqlStrlen30(zCsr);
+		}
+		*zCsr = '\0';
+	} else if (f & MEM_Str) {
+		int j, k;
+		zBuf[0] = ' ';
+		if (f & MEM_Dyn) {
+			zBuf[1] = 'z';
+			assert((f & (MEM_Static|MEM_Ephem))==0);
+		} else if (f & MEM_Static) {
+			zBuf[1] = 't';
+			assert((f & (MEM_Dyn|MEM_Ephem))==0);
+		} else if (f & MEM_Ephem) {
+			zBuf[1] = 'e';
+			assert((f & (MEM_Static|MEM_Dyn))==0);
+		} else {
+			zBuf[1] = 's';
+		}
+		k = 2;
+		sql_snprintf(100, &zBuf[k], "%d", pMem->n);
+		k += sqlStrlen30(&zBuf[k]);
+		zBuf[k++] = '[';
+		for(j=0; j<15 && j<pMem->n; j++) {
+			u8 c = pMem->z[j];
+			if (c>=0x20 && c<0x7f) {
+				zBuf[k++] = c;
+			} else {
+				zBuf[k++] = '.';
+			}
+		}
+		zBuf[k++] = ']';
+		sql_snprintf(100,&zBuf[k],"(8)");
+		k += sqlStrlen30(&zBuf[k]);
+		zBuf[k++] = 0;
+	}
+}
+
+/*
+ * Print the value of a register for tracing purposes:
+ */
+static void
+memTracePrint(Mem *p)
+{
+	if (p->flags & MEM_Undefined) {
+		printf(" undefined");
+	} else if (p->flags & MEM_Null) {
+		printf(" NULL");
+	} else if ((p->flags & (MEM_Int|MEM_Str))==(MEM_Int|MEM_Str)) {
+		printf(" si:%lld", p->u.i);
+	} else if (p->flags & MEM_Int) {
+		printf(" i:%lld", p->u.i);
+	} else if (p->flags & MEM_UInt) {
+		printf(" u:%"PRIu64"", p->u.u);
+	} else if (p->flags & MEM_Real) {
+		printf(" r:%g", p->u.r);
+	} else if (p->flags & MEM_Bool) {
+		printf(" bool:%s", SQL_TOKEN_BOOLEAN(p->u.b));
+	} else {
+		char zBuf[200];
+		sqlVdbeMemPrettyPrint(p, zBuf);
+		printf(" %s", zBuf);
+	}
+	if (p->flags & MEM_Subtype) printf(" subtype=0x%02x", p->subtype);
+}
+
+void
+registerTrace(int iReg, Mem *p) {
+	printf("REG[%d] = ", iReg);
+	memTracePrint(p);
+	printf("\n");
+}
+#endif
+
+int
+mem_apply_numeric_type(struct Mem *record)
+{
+	if ((record->flags & MEM_Str) == 0)
+		return -1;
+	int64_t integer_value;
+	bool is_neg;
+	if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
+		mem_set_int(record, integer_value, is_neg);
+		return 0;
+	}
+	double float_value;
+	if (sqlAtoF(record->z, &float_value, record->n) == 0)
+		return -1;
+	mem_set_double(record, float_value);
+	return 0;
+}
+
+/*
+ * Convert pMem so that it is of type MEM_Real.
+ * Invalidate any prior representations.
+ */
+int
+sqlVdbeMemRealify(Mem * pMem)
+{
+	assert(EIGHT_BYTE_ALIGNMENT(pMem));
+	double v;
+	if (sqlVdbeRealValue(pMem, &v))
+		return -1;
+	mem_set_double(pMem, v);
+	return 0;
+}
+
+int
+vdbe_mem_numerify(struct Mem *mem)
+{
+	if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
+		return 0;
+	if ((mem->flags & MEM_Bool) != 0) {
+		mem->u.u = mem->u.b;
+		MemSetTypeFlag(mem, MEM_UInt);
+		return 0;
+	}
+	assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
+	bool is_neg;
+	int64_t i;
+	if (sql_atoi64(mem->z, &i, &is_neg, mem->n) == 0) {
+		mem_set_int(mem, i, is_neg);
+	} else {
+		double d;
+		if (sqlAtoF(mem->z, &d, mem->n) == 0)
+			return -1;
+		mem_set_double(mem, d);
+	}
+	return 0;
+}
+
+/*
+ * Cast the datatype of the value in pMem according to the type
+ * @type.  Casting is different from applying type in that a cast
+ * is forced.  In other words, the value is converted into the desired
+ * type even if that results in loss of data.  This routine is
+ * used (for example) to implement the SQL "cast()" operator.
+ */
+int
+sqlVdbeMemCast(Mem * pMem, enum field_type type)
+{
+	assert(type < field_type_MAX);
+	if (pMem->flags & MEM_Null)
+		return 0;
+	switch (type) {
+	case FIELD_TYPE_SCALAR:
+		return 0;
+	case FIELD_TYPE_BOOLEAN:
+		if ((pMem->flags & MEM_Int) != 0) {
+			mem_set_bool(pMem, pMem->u.i);
+			return 0;
+		}
+		if ((pMem->flags & MEM_UInt) != 0) {
+			mem_set_bool(pMem, pMem->u.u);
+			return 0;
+		}
+		if ((pMem->flags & MEM_Real) != 0) {
+			mem_set_bool(pMem, pMem->u.r);
+			return 0;
+		}
+		if ((pMem->flags & MEM_Str) != 0) {
+			bool value;
+			if (str_cast_to_boolean(pMem->z, &value) != 0)
+				return -1;
+			mem_set_bool(pMem, value);
+			return 0;
+		}
+		if ((pMem->flags & MEM_Bool) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_INTEGER:
+	case FIELD_TYPE_UNSIGNED:
+		if ((pMem->flags & (MEM_Blob | MEM_Str)) != 0) {
+			bool is_neg;
+			int64_t val;
+			if (sql_atoi64(pMem->z, &val, &is_neg, pMem->n) != 0)
+				return -1;
+			if (type == FIELD_TYPE_UNSIGNED && is_neg)
+				return -1;
+			mem_set_int(pMem, val, is_neg);
+			return 0;
+		}
+		if ((pMem->flags & MEM_Bool) != 0) {
+			pMem->u.u = pMem->u.b;
+			MemSetTypeFlag(pMem, MEM_UInt);
+			return 0;
+		}
+		if ((pMem->flags & MEM_Real) != 0) {
+			double d;
+			if (sqlVdbeRealValue(pMem, &d) != 0)
+				return -1;
+			if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
+				mem_set_int(pMem, d, d <= -1);
+				return 0;
+			}
+			if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
+				mem_set_u64(pMem, d);
+				return 0;
+			}
+			return -1;
+		}
+		if (type == FIELD_TYPE_UNSIGNED &&
+		    (pMem->flags & MEM_UInt) == 0)
+			return -1;
+		return 0;
+	case FIELD_TYPE_DOUBLE:
+		return sqlVdbeMemRealify(pMem);
+	case FIELD_TYPE_NUMBER:
+		return vdbe_mem_numerify(pMem);
+	case FIELD_TYPE_VARBINARY:
+		if ((pMem->flags & MEM_Blob) != 0)
+			return 0;
+		if ((pMem->flags & MEM_Str) != 0) {
+			MemSetTypeFlag(pMem, MEM_Str);
+			return 0;
+		}
+		return -1;
+	default:
+		assert(type == FIELD_TYPE_STRING);
+		assert(MEM_Str == (MEM_Blob >> 3));
+		if ((pMem->flags & MEM_Bool) != 0) {
+			const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
+			sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
+					 SQL_TRANSIENT);
+			return 0;
+		}
+		pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
+			sql_value_apply_type(pMem, FIELD_TYPE_STRING);
+		assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
+		pMem->flags &=
+			~(MEM_Int | MEM_UInt | MEM_Real | MEM_Blob | MEM_Zero);
+		return 0;
+	}
+}
+
+/*
+ * The MEM structure is already a MEM_Real.  Try to also make it a
+ * MEM_Int if we can.
+ */
+int
+mem_apply_integer_type(Mem *pMem)
+{
+	int rc;
+	i64 ix;
+	assert(pMem->flags & MEM_Real);
+	assert(EIGHT_BYTE_ALIGNMENT(pMem));
+
+	if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
+		mem_set_int(pMem, ix, pMem->u.r <= -1);
+	return rc;
+}
+
+/*
+ * Add MEM_Str to the set of representations for the given Mem.  Numbers
+ * are converted using sql_snprintf().  Converting a BLOB to a string
+ * is a no-op.
+ *
+ * Existing representations MEM_Int and MEM_Real are invalidated if
+ * bForce is true but are retained if bForce is false.
+ *
+ * A MEM_Null value will never be passed to this function. This function is
+ * used for converting values to text for returning to the user (i.e. via
+ * sql_value_text()), or for ensuring that values to be used as btree
+ * keys are strings. In the former case a NULL pointer is returned the
+ * user and the latter is an internal programming error.
+ */
+int
+sqlVdbeMemStringify(Mem * pMem)
+{
+	int fg = pMem->flags;
+	int nByte = 32;
+
+	if ((fg & (MEM_Null | MEM_Str | MEM_Blob)) != 0 &&
+	    !mem_has_msgpack_subtype(pMem))
+		return 0;
+
+	assert(!(fg & MEM_Zero));
+	assert((fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool |
+		      MEM_Blob)) != 0);
+	assert(EIGHT_BYTE_ALIGNMENT(pMem));
+
+	/*
+	 * In case we have ARRAY/MAP we should save decoded value
+	 * before clearing pMem->z.
+	 */
+	char *value = NULL;
+	if (mem_has_msgpack_subtype(pMem)) {
+		const char *value_str = mp_str(pMem->z);
+		nByte = strlen(value_str) + 1;
+		value = region_alloc(&fiber()->gc, nByte);
+		memcpy(value, value_str, nByte);
+	}
+
+	if (sqlVdbeMemClearAndResize(pMem, nByte)) {
+		return -1;
+	}
+	if (fg & MEM_Int) {
+		sql_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
+		pMem->flags &= ~MEM_Int;
+	} else if ((fg & MEM_UInt) != 0) {
+		sql_snprintf(nByte, pMem->z, "%llu", pMem->u.u);
+		pMem->flags &= ~MEM_UInt;
+	} else if ((fg & MEM_Bool) != 0) {
+		sql_snprintf(nByte, pMem->z, "%s",
+			     SQL_TOKEN_BOOLEAN(pMem->u.b));
+		pMem->flags &= ~MEM_Bool;
+	} else if (mem_has_msgpack_subtype(pMem)) {
+		sql_snprintf(nByte, pMem->z, "%s", value);
+		pMem->flags &= ~MEM_Subtype;
+		pMem->subtype = SQL_SUBTYPE_NO;
+	} else {
+		assert(fg & MEM_Real);
+		sql_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
+		pMem->flags &= ~MEM_Real;
+	}
+	pMem->n = sqlStrlen30(pMem->z);
+	pMem->flags |= MEM_Str | MEM_Term;
+	return 0;
+}
+
+/*
+ * Make sure the given Mem is \u0000 terminated.
+ */
+int
+sqlVdbeMemNulTerminate(Mem * pMem)
+{
+	testcase((pMem->flags & (MEM_Term | MEM_Str)) == (MEM_Term | MEM_Str));
+	testcase((pMem->flags & (MEM_Term | MEM_Str)) == 0);
+	if ((pMem->flags & (MEM_Term | MEM_Str)) != MEM_Str) {
+		return 0;	/* Nothing to do */
+	} else {
+		return vdbeMemAddTerminator(pMem);
+	}
+}
+
+/*
+ * If the given Mem* has a zero-filled tail, turn it into an ordinary
+ * blob stored in dynamically allocated space.
+ */
+int
+sqlVdbeMemExpandBlob(Mem * pMem)
+{
+	int nByte;
+	assert(pMem->flags & MEM_Zero);
+	assert(pMem->flags & MEM_Blob);
+
+	/* Set nByte to the number of bytes required to store the expanded blob. */
+	nByte = pMem->n + pMem->u.nZero;
+	if (nByte <= 0) {
+		nByte = 1;
+	}
+	if (sqlVdbeMemGrow(pMem, nByte, 1)) {
+		return -1;
+	}
+
+	memset(&pMem->z[pMem->n], 0, pMem->u.nZero);
+	pMem->n += pMem->u.nZero;
+	pMem->flags &= ~(MEM_Zero | MEM_Term);
+	return 0;
+}
+
+/*
+ * Exported version of mem_apply_type(). This one works on sql_value*,
+ * not the internal Mem* type.
+ */
+void
+sql_value_apply_type(
+	sql_value *pVal,
+	enum field_type type)
+{
+	mem_apply_type((Mem *) pVal, type);
+}
+
+int
+mem_apply_type(struct Mem *record, enum field_type type)
+{
+	if ((record->flags & MEM_Null) != 0)
+		return 0;
+	assert(type < field_type_MAX);
+	switch (type) {
+	case FIELD_TYPE_INTEGER:
+	case FIELD_TYPE_UNSIGNED:
+		if ((record->flags & (MEM_Bool | MEM_Blob)) != 0)
+			return -1;
+		if ((record->flags & MEM_UInt) == MEM_UInt)
+			return 0;
+		if ((record->flags & MEM_Real) == MEM_Real) {
+			double d = record->u.r;
+			if (d >= 0) {
+				if (double_compare_uint64(d, UINT64_MAX,
+							  1) > 0)
+					return 0;
+				if ((double)(uint64_t)d == d)
+					mem_set_u64(record, (uint64_t)d);
+			} else {
+				if (double_compare_nint64(d, INT64_MIN, 1) < 0)
+					return 0;
+				if ((double)(int64_t)d == d)
+					mem_set_int(record, (int64_t)d, true);
+			}
+			return 0;
+		}
+		if ((record->flags & MEM_Str) != 0) {
+			bool is_neg;
+			int64_t i;
+			if (sql_atoi64(record->z, &i, &is_neg, record->n) != 0)
+				return -1;
+			mem_set_int(record, i, is_neg);
+		}
+		if ((record->flags & MEM_Int) == MEM_Int) {
+			if (type == FIELD_TYPE_UNSIGNED)
+				return -1;
+			return 0;
+		}
+		return 0;
+	case FIELD_TYPE_BOOLEAN:
+		if ((record->flags & MEM_Bool) == MEM_Bool)
+			return 0;
+		return -1;
+	case FIELD_TYPE_NUMBER:
+		if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
+			return 0;
+		return sqlVdbeMemRealify(record);
+	case FIELD_TYPE_DOUBLE:
+		if ((record->flags & MEM_Real) != 0)
+			return 0;
+		return sqlVdbeMemRealify(record);
+	case FIELD_TYPE_STRING:
+		/*
+		 * Only attempt the conversion to TEXT if there is
+		 * an integer or real representation (BLOB and
+		 * NULL do not get converted).
+		 */
+		if ((record->flags & MEM_Str) == 0 &&
+		    (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
+			sqlVdbeMemStringify(record);
+		record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
+		return 0;
+	case FIELD_TYPE_VARBINARY:
+		if ((record->flags & MEM_Blob) == 0)
+			return -1;
+		return 0;
+	case FIELD_TYPE_SCALAR:
+		/* Can't cast MAP and ARRAY to scalar types. */
+		if ((record->flags & MEM_Subtype) != 0 &&
+		    record->subtype == SQL_SUBTYPE_MSGPACK) {
+			assert(mp_typeof(*record->z) == MP_MAP ||
+			       mp_typeof(*record->z) == MP_ARRAY);
+			return -1;
+		}
+		return 0;
+	case FIELD_TYPE_MAP:
+		if ((record->flags & MEM_Subtype) != 0 &&
+		    record->subtype == SQL_SUBTYPE_MSGPACK &&
+		    mp_typeof(*record->z) == MP_MAP)
+			return 0;
+		return -1;
+	case FIELD_TYPE_ARRAY:
+		if ((record->flags & MEM_Subtype) != 0 &&
+		    record->subtype == SQL_SUBTYPE_MSGPACK &&
+		    mp_typeof(*record->z) == MP_ARRAY)
+			return 0;
+		return -1;
+	case FIELD_TYPE_ANY:
+		return 0;
+	default:
+		return -1;
+	}
+}
+
+/**
+ * Convert the numeric value contained in MEM to double.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+static int
+mem_convert_to_double(struct Mem *mem)
+{
+	if ((mem->flags & MEM_Real) != 0)
+		return 0;
+	if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
+		return -1;
+	double d;
+	if ((mem->flags & MEM_Int) != 0)
+		d = (double)mem->u.i;
+	else
+		d = (double)mem->u.u;
+	mem_set_double(mem, d);
+	return 0;
+}
+
+/**
+ * Convert the numeric value contained in MEM to unsigned.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+static int
+mem_convert_to_unsigned(struct Mem *mem)
+{
+	if ((mem->flags & MEM_UInt) != 0)
+		return 0;
+	if ((mem->flags & MEM_Int) != 0)
+		return -1;
+	if ((mem->flags & MEM_Real) == 0)
+		return -1;
+	double d = mem->u.r;
+	if (d < 0.0 || d >= (double)UINT64_MAX)
+		return -1;
+	mem_set_u64(mem, (uint64_t) d);
+	return 0;
+}
+
+/**
+ * Convert the numeric value contained in MEM to integer.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+static int
+mem_convert_to_integer(struct Mem *mem)
+{
+	if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
+		return 0;
+	if ((mem->flags & MEM_Real) == 0)
+		return -1;
+	double d = mem->u.r;
+	if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
+		return -1;
+	if (d < (double)INT64_MAX)
+		mem_set_int(mem, (int64_t) d, d < 0);
+	else
+		mem_set_int(mem, (uint64_t) d, false);
+	return 0;
+}
+
+int
+mem_convert_to_numeric(struct Mem *mem, enum field_type type)
+{
+	assert(mp_type_is_numeric(mem_mp_type(mem)) &&
+	       sql_type_is_numeric(type));
+	assert(type != FIELD_TYPE_NUMBER);
+	if (type == FIELD_TYPE_DOUBLE)
+		return mem_convert_to_double(mem);
+	if (type == FIELD_TYPE_UNSIGNED)
+		return mem_convert_to_unsigned(mem);
+	assert(type == FIELD_TYPE_INTEGER);
+	return mem_convert_to_integer(mem);
+}
+
+/*
+ * Make sure pMem->z points to a writable allocation of at least
+ * min(n,32) bytes.
+ *
+ * If the bPreserve argument is true, then copy of the content of
+ * pMem->z into the new allocation.  pMem must be either a string or
+ * blob if bPreserve is true.  If bPreserve is false, any prior content
+ * in pMem->z is discarded.
+ */
+SQL_NOINLINE int
+sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
+{
+	assert(sqlVdbeCheckMemInvariants(pMem));
+	testcase(pMem->db == 0);
+
+	/* If the bPreserve flag is set to true, then the memory cell must already
+	 * contain a valid string or blob value.
+	 */
+	assert(bPreserve == 0 || pMem->flags & (MEM_Blob | MEM_Str));
+	testcase(bPreserve && pMem->z == 0);
+
+	assert(pMem->szMalloc == 0 ||
+	       pMem->szMalloc == sqlDbMallocSize(pMem->db, pMem->zMalloc));
+	if (pMem->szMalloc < n) {
+		if (n < 32)
+			n = 32;
+		if (bPreserve && pMem->szMalloc > 0 && pMem->z == pMem->zMalloc) {
+			pMem->z = pMem->zMalloc =
+			    sqlDbReallocOrFree(pMem->db, pMem->z, n);
+			bPreserve = 0;
+		} else {
+			if (pMem->szMalloc > 0)
+				sqlDbFree(pMem->db, pMem->zMalloc);
+			pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
+		}
+		if (pMem->zMalloc == 0) {
+			sqlVdbeMemSetNull(pMem);
+			pMem->z = 0;
+			pMem->szMalloc = 0;
+			return -1;
+		} else {
+			pMem->szMalloc = sqlDbMallocSize(pMem->db,
+							 pMem->zMalloc);
+		}
+	}
+
+	if (bPreserve && pMem->z && pMem->z != pMem->zMalloc) {
+		memcpy(pMem->zMalloc, pMem->z, pMem->n);
+	}
+	if ((pMem->flags & MEM_Dyn) != 0) {
+		assert(pMem->xDel != 0 && pMem->xDel != SQL_DYNAMIC);
+		pMem->xDel((void *)(pMem->z));
+	}
+
+	pMem->z = pMem->zMalloc;
+	pMem->flags &= ~(MEM_Dyn | MEM_Ephem | MEM_Static);
+	return 0;
+}
+
+/*
+ * Change the pMem->zMalloc allocation to be at least szNew bytes.
+ * If pMem->zMalloc already meets or exceeds the requested size, this
+ * routine is a no-op.
+ *
+ * Any prior string or blob content in the pMem object may be discarded.
+ * The pMem->xDel destructor is called, if it exists.  Though MEM_Str
+ * and MEM_Blob values may be discarded, MEM_Int, MEM_Real, and MEM_Null
+ * values are preserved.
+ *
+ * Return 0 on success or -1 if unable to complete the resizing.
+ */
+int
+sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
+{
+	assert(szNew > 0);
+	assert((pMem->flags & MEM_Dyn) == 0 || pMem->szMalloc == 0);
+	if (pMem->szMalloc < szNew) {
+		return sqlVdbeMemGrow(pMem, szNew, 0);
+	}
+	assert((pMem->flags & MEM_Dyn) == 0);
+	pMem->z = pMem->zMalloc;
+	pMem->flags &= (MEM_Null | MEM_Int | MEM_Real);
+	return 0;
+}
+
+void
+mem_set_bool(struct Mem *mem, bool value)
+{
+	sqlVdbeMemSetNull(mem);
+	mem->u.b = value;
+	mem->flags = MEM_Bool;
+	mem->field_type = FIELD_TYPE_BOOLEAN;
+}
+
+void
+mem_set_ptr(struct Mem *mem, void *ptr)
+{
+	sqlVdbeMemRelease(mem);
+	mem->flags = MEM_Ptr;
+	mem->u.p = ptr;
+}
+
+void
+mem_set_i64(struct Mem *mem, int64_t value)
+{
+	if (VdbeMemDynamic(mem))
+		sqlVdbeMemSetNull(mem);
+	mem->u.i = value;
+	int flag = value < 0 ? MEM_Int : MEM_UInt;
+	MemSetTypeFlag(mem, flag);
+	mem->field_type = FIELD_TYPE_INTEGER;
+}
+
+void
+mem_set_u64(struct Mem *mem, uint64_t value)
+{
+	if (VdbeMemDynamic(mem))
+		sqlVdbeMemSetNull(mem);
+	mem->u.u = value;
+	MemSetTypeFlag(mem, MEM_UInt);
+	mem->field_type = FIELD_TYPE_UNSIGNED;
+}
+
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
+{
+	if (VdbeMemDynamic(mem))
+		sqlVdbeMemSetNull(mem);
+	if (is_neg) {
+		assert(value < 0);
+		mem->u.i = value;
+		MemSetTypeFlag(mem, MEM_Int);
+	} else {
+		mem->u.u = value;
+		MemSetTypeFlag(mem, MEM_UInt);
+	}
+	mem->field_type = FIELD_TYPE_INTEGER;
+}
+
+void
+mem_set_double(struct Mem *mem, double value)
+{
+	sqlVdbeMemSetNull(mem);
+	if (sqlIsNaN(value))
+		return;
+	mem->u.r = value;
+	MemSetTypeFlag(mem, MEM_Real);
+	mem->field_type = FIELD_TYPE_DOUBLE;
+}
+
+/*
+ * Change the value of a Mem to be a string or a BLOB.
+ *
+ * The memory management strategy depends on the value of the xDel
+ * parameter. If the value passed is SQL_TRANSIENT, then the
+ * string is copied into a (possibly existing) buffer managed by the
+ * Mem structure. Otherwise, any existing buffer is freed and the
+ * pointer copied.
+ *
+ * If the string is too large (if it exceeds the SQL_LIMIT_LENGTH
+ * size limit) then no memory allocation occurs.  If the string can be
+ * stored without allocating memory, then it is.  If a memory allocation
+ * is required to store the string, then value of pMem is unchanged.  In
+ * either case, error is returned.
+ */
+int
+sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
+		     const char *z,	/* String pointer */
+		     int n,	/* Bytes in string, or negative */
+		     u8 not_blob,	/* Encoding of z.  0 for BLOBs */
+		     void (*xDel) (void *)	/* Destructor function */
+    )
+{
+	int nByte = n;		/* New value for pMem->n */
+	int iLimit;		/* Maximum allowed string or blob size */
+	u16 flags = 0;		/* New value for pMem->flags */
+
+	/* If z is a NULL pointer, set pMem to contain an SQL NULL. */
+	if (!z) {
+		sqlVdbeMemSetNull(pMem);
+		return 0;
+	}
+
+	if (pMem->db) {
+		iLimit = pMem->db->aLimit[SQL_LIMIT_LENGTH];
+	} else {
+		iLimit = SQL_MAX_LENGTH;
+	}
+	flags = (not_blob == 0 ? MEM_Blob : MEM_Str);
+	if (nByte < 0) {
+		assert(not_blob != 0);
+		nByte = sqlStrlen30(z);
+		if (nByte > iLimit)
+			nByte = iLimit + 1;
+		flags |= MEM_Term;
+	}
+
+	/* The following block sets the new values of Mem.z and Mem.xDel. It
+	 * also sets a flag in local variable "flags" to indicate the memory
+	 * management (one of MEM_Dyn or MEM_Static).
+	 */
+	if (xDel == SQL_TRANSIENT) {
+		int nAlloc = nByte;
+		if (flags & MEM_Term) {
+			nAlloc += 1; //SQL_UTF8
+		}
+		if (nByte > iLimit) {
+			diag_set(ClientError, ER_SQL_EXECUTE, "string or binary"\
+				 "string is too big");
+			return -1;
+		}
+		testcase(nAlloc == 0);
+		testcase(nAlloc == 31);
+		testcase(nAlloc == 32);
+		if (sqlVdbeMemClearAndResize(pMem, MAX(nAlloc, 32))) {
+			return -1;
+		}
+		memcpy(pMem->z, z, nAlloc);
+	} else if (xDel == SQL_DYNAMIC) {
+		sqlVdbeMemRelease(pMem);
+		pMem->zMalloc = pMem->z = (char *)z;
+		pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
+	} else {
+		sqlVdbeMemRelease(pMem);
+		pMem->z = (char *)z;
+		pMem->xDel = xDel;
+		flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
+	}
+
+	pMem->n = nByte;
+	pMem->flags = flags;
+	assert((pMem->flags & (MEM_Str | MEM_Blob)) != 0);
+	if ((pMem->flags & MEM_Str) != 0)
+		pMem->field_type = FIELD_TYPE_STRING;
+	else
+		pMem->field_type = FIELD_TYPE_VARBINARY;
+
+	if (nByte > iLimit) {
+		diag_set(ClientError, ER_SQL_EXECUTE, "string or binary string"\
+			 "is too big");
+		return -1;
+	}
+
+	return 0;
+}
+
+/*
+ * Initialize bulk memory to be a consistent Mem object.
+ *
+ * The minimum amount of initialization feasible is performed.
+ */
+void
+sqlVdbeMemInit(Mem * pMem, sql * db, u32 flags)
+{
+	assert((flags & ~MEM_TypeMask) == 0);
+	pMem->flags = flags;
+	pMem->db = db;
+	pMem->szMalloc = 0;
+	pMem->field_type = field_type_MAX;
+}
+
+/*
+ * Delete any previous value and set the value stored in *pMem to NULL.
+ *
+ * This routine calls the Mem.xDel destructor to dispose of values that
+ * require the destructor.  But it preserves the Mem.zMalloc memory allocation.
+ * To free all resources, use sqlVdbeMemRelease(), which both calls this
+ * routine to invoke the destructor and deallocates Mem.zMalloc.
+ *
+ * Use this routine to reset the Mem prior to insert a new value.
+ *
+ * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
+ */
+void
+sqlVdbeMemSetNull(Mem * pMem)
+{
+	if (VdbeMemDynamic(pMem)) {
+		vdbeMemClearExternAndSetNull(pMem);
+	} else {
+		pMem->flags = MEM_Null;
+	}
+}
+
+/*
+ * Delete any previous value and set the value to be a BLOB of length
+ * n containing all zeros.
+ */
+void
+sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
+{
+	sqlVdbeMemRelease(pMem);
+	pMem->flags = MEM_Blob | MEM_Zero;
+	pMem->n = 0;
+	if (n < 0)
+		n = 0;
+	pMem->u.nZero = n;
+	pMem->z = 0;
+}
+
+/*
+ * Change the string value of an sql_value object
+ */
+void
+sqlValueSetStr(sql_value * v,	/* Value to be set */
+		   int n,	/* Length of string z */
+		   const void *z,	/* Text of the new string */
+		   void (*xDel) (void *)	/* Destructor for the string */
+    )
+{
+	if (v)
+		sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
+}
+
+void
+sqlValueSetNull(sql_value * p)
+{
+	sqlVdbeMemSetNull((Mem *) p);
+}
+
+/*
+ * Free an sql_value object
+ */
+void
+sqlValueFree(sql_value * v)
+{
+	if (!v)
+		return;
+	sqlVdbeMemRelease((Mem *) v);
+	sqlDbFree(((Mem *) v)->db, v);
+}
+
+/*
+ * Create a new sql_value object.
+ */
+sql_value *
+sqlValueNew(sql * db)
+{
+	Mem *p = sqlDbMallocZero(db, sizeof(*p));
+	if (p) {
+		p->flags = MEM_Null;
+		p->db = db;
+	}
+	return p;
+}
+
+void
+initMemArray(Mem * p, int N, sql * db, u32 flags)
+{
+	while ((N--) > 0) {
+		p->db = db;
+		p->flags = flags;
+		p->szMalloc = 0;
+		p->field_type = field_type_MAX;
+#ifdef SQL_DEBUG
+		p->pScopyFrom = 0;
+#endif
+		p++;
+	}
+}
+
+void
+releaseMemArray(Mem * p, int N)
+{
+	if (p && N) {
+		Mem *pEnd = &p[N];
+		sql *db = p->db;
+		do {
+			assert((&p[1]) == pEnd || p[0].db == p[1].db);
+			assert(sqlVdbeCheckMemInvariants(p));
+
+			/* This block is really an inlined version of sqlVdbeMemRelease()
+			 * that takes advantage of the fact that the memory cell value is
+			 * being set to NULL after releasing any dynamic resources.
+			 *
+			 * The justification for duplicating code is that according to
+			 * callgrind, this causes a certain test case to hit the CPU 4.7
+			 * percent less (x86 linux, gcc version 4.1.2, -O6) than if
+			 * sqlMemRelease() were called from here. With -O2, this jumps
+			 * to 6.6 percent. The test case is inserting 1000 rows into a table
+			 * with no indexes using a single prepared INSERT statement, bind()
+			 * and reset(). Inserts are grouped into a transaction.
+			 */
+			testcase(p->flags & MEM_Agg);
+			testcase(p->flags & MEM_Dyn);
+			testcase(p->flags & MEM_Frame);
+			if (p->
+			    flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
+				sqlVdbeMemRelease(p);
+			} else if (p->szMalloc) {
+				sqlDbFree(db, p->zMalloc);
+				p->szMalloc = 0;
+			}
+
+			p->flags = MEM_Undefined;
+		} while ((++p) < pEnd);
+	}
+}
+
+int
+mem_value_bool(const struct Mem *mem, bool *b)
+{
+	if ((mem->flags  & MEM_Bool) != 0) {
+		*b = mem->u.b;
+		return 0;
+	}
+	return -1;
+}
+
+/*
+ * Return some kind of integer value which is the best we can do
+ * at representing the value that *pMem describes as an integer.
+ * If pMem is an integer, then the value is exact.  If pMem is
+ * a floating-point then the value returned is the integer part.
+ * If pMem is a string or blob, then we make an attempt to convert
+ * it into an integer and return that.  If pMem represents an
+ * an SQL-NULL value, return 0.
+ *
+ * If pMem represents a string value, its encoding might be changed.
+ */
+int
+sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
+{
+	int flags;
+	assert(EIGHT_BYTE_ALIGNMENT(pMem));
+	flags = pMem->flags;
+	if (flags & MEM_Int) {
+		*i = pMem->u.i;
+		*is_neg = true;
+		return 0;
+	} else if (flags & MEM_UInt) {
+		*i = pMem->u.u;
+		*is_neg = false;
+		return 0;
+	} else if (flags & MEM_Real) {
+		*is_neg = pMem->u.r < 0;
+		return doubleToInt64(pMem->u.r, i);
+	} else if (flags & (MEM_Str)) {
+		assert(pMem->z || pMem->n == 0);
+		if (sql_atoi64(pMem->z, i, is_neg, pMem->n) == 0)
+			return 0;
+	}
+	return -1;
+}
+
+/*
+ * Return the best representation of pMem that we can get into a
+ * double.  If pMem is already a double or an integer, return its
+ * value.  If it is a string or blob, try to convert it to a double.
+ * If it is a NULL, return 0.0.
+ */
+int
+sqlVdbeRealValue(Mem * pMem, double *v)
+{
+	assert(EIGHT_BYTE_ALIGNMENT(pMem));
+	if (pMem->flags & MEM_Real) {
+		*v = pMem->u.r;
+		return 0;
+	} else if (pMem->flags & MEM_Int) {
+		*v = (double)pMem->u.i;
+		return 0;
+	} else if ((pMem->flags & MEM_UInt) != 0) {
+		*v = (double)pMem->u.u;
+		return 0;
+	} else if (pMem->flags & MEM_Str) {
+		if (sqlAtoF(pMem->z, v, pMem->n))
+			return 0;
+	}
+	return -1;
+}
+
+/**************************** sql_value_  ******************************
+ * The following routines extract information from a Mem or sql_value
+ * structure.
+ */
+const void *
+sql_value_blob(sql_value * pVal)
+{
+	Mem *p = (Mem *) pVal;
+	if (p->flags & (MEM_Blob | MEM_Str)) {
+		if (ExpandBlob(p) != 0) {
+			assert(p->flags == MEM_Null && p->z == 0);
+			return 0;
+		}
+		p->flags |= MEM_Blob;
+		return p->n ? p->z : 0;
+	} else {
+		return sql_value_text(pVal);
+	}
+}
+
+int
+sql_value_bytes(sql_value * pVal)
+{
+	return sqlValueBytes(pVal);
+}
+
+double
+sql_value_double(sql_value * pVal)
+{
+	double v = 0.0;
+	sqlVdbeRealValue((Mem *) pVal, &v);
+	return v;
+}
+
+bool
+sql_value_boolean(sql_value *val)
+{
+	bool b = false;
+	int rc = mem_value_bool((struct Mem *) val, &b);
+	assert(rc == 0);
+	(void) rc;
+	return b;
+}
+
+int
+sql_value_int(sql_value * pVal)
+{
+	int64_t i = 0;
+	bool is_neg;
+	sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
+	return (int)i;
+}
+
+sql_int64
+sql_value_int64(sql_value * pVal)
+{
+	int64_t i = 0;
+	bool unused;
+	sqlVdbeIntValue((Mem *) pVal, &i, &unused);
+	return i;
+}
+
+uint64_t
+sql_value_uint64(sql_value *val)
+{
+	int64_t i = 0;
+	bool is_neg;
+	sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
+	assert(!is_neg);
+	return i;
+}
+
+const unsigned char *
+sql_value_text(sql_value * pVal)
+{
+	return (const unsigned char *)sqlValueText(pVal);
+}
+
+/* This function is only available internally, it is not part of the
+ * external API. It works in a similar way to sql_value_text(),
+ * except the data returned is in the encoding specified by the second
+ * parameter, which must be one of SQL_UTF16BE, SQL_UTF16LE or
+ * SQL_UTF8.
+ *
+ * (2006-02-16:)  The enc value can be or-ed with SQL_UTF16_ALIGNED.
+ * If that is the case, then the result must be aligned on an even byte
+ * boundary.
+ */
+const void *
+sqlValueText(sql_value * pVal)
+{
+	if (!pVal)
+		return 0;
+	if ((pVal->flags & (MEM_Str | MEM_Term)) == (MEM_Str | MEM_Term)) {
+		return pVal->z;
+	}
+	if (pVal->flags & MEM_Null) {
+		return 0;
+	}
+	return valueToText(pVal);
+}
+
+const char *
+sql_value_to_diag_str(sql_value *value)
+{
+	enum mp_type mp_type = sql_value_type(value);
+	if (mp_type_is_bloblike(mp_type)) {
+		if (mem_has_msgpack_subtype(value))
+			return sqlValueText(value);
+		return "varbinary";
+	}
+	return sqlValueText(value);
+}
+
+enum sql_subtype
+sql_value_subtype(sql_value * pVal)
+{
+	return (pVal->flags & MEM_Subtype) != 0 ? pVal->subtype : SQL_SUBTYPE_NO;
+}
+
+/*
+ * Return a pointer to static memory containing an SQL NULL value.
+ */
+const Mem *
+columnNullValue(void)
+{
+	/* Even though the Mem structure contains an element
+	 * of type i64, on certain architectures (x86) with certain compiler
+	 * switches (-Os), gcc may align this Mem object on a 4-byte boundary
+	 * instead of an 8-byte one. This all works fine, except that when
+	 * running with SQL_DEBUG defined the sql code sometimes assert()s
+	 * that a Mem structure is located on an 8-byte boundary. To prevent
+	 * these assert()s from failing, when building with SQL_DEBUG defined
+	 * using gcc, we force nullMem to be 8-byte aligned using the magical
+	 * __attribute__((aligned(8))) macro.
+	 */
+	static const Mem nullMem
+#if defined(SQL_DEBUG) && defined(__GNUC__)
+	    __attribute__ ((aligned(8)))
+#endif
+	    = {
+		/* .u          = */  {
+		0},
+		    /* .flags      = */ (u16) MEM_Null,
+		    /* .eSubtype   = */ (u8) 0,
+		    /* .field_type = */ field_type_MAX,
+		    /* .n          = */ (int)0,
+		    /* .z          = */ (char *)0,
+		    /* .zMalloc    = */ (char *)0,
+		    /* .szMalloc   = */ (int)0,
+		    /* .uTemp      = */ (u32) 0,
+		    /* .db         = */ (sql *) 0,
+		    /* .xDel       = */ (void (*)(void *))0,
+#ifdef SQL_DEBUG
+		    /* .pScopyFrom = */ (Mem *) 0,
+		    /* .pFiller    = */ (void *)0,
+#endif
+	};
+	return &nullMem;
+}
+
+/*
+ * Return true if the Mem object contains a TEXT or BLOB that is
+ * too large - whose size exceeds SQL_MAX_LENGTH.
+ */
+int
+sqlVdbeMemTooBig(Mem * p)
+{
+	assert(p->db != 0);
+	if (p->flags & (MEM_Str | MEM_Blob)) {
+		int n = p->n;
+		if (p->flags & MEM_Zero) {
+			n += p->u.nZero;
+		}
+		return n > p->db->aLimit[SQL_LIMIT_LENGTH];
+	}
+	return 0;
+}
+
+/*
+ * Compare two blobs.  Return negative, zero, or positive if the first
+ * is less than, equal to, or greater than the second, respectively.
+ * If one blob is a prefix of the other, then the shorter is the lessor.
+ */
+static SQL_NOINLINE int
+sqlBlobCompare(const Mem * pB1, const Mem * pB2)
+{
+	int c;
+	int n1 = pB1->n;
+	int n2 = pB2->n;
+
+	/* It is possible to have a Blob value that has some non-zero content
+	 * followed by zero content.  But that only comes up for Blobs formed
+	 * by the OP_MakeRecord opcode, and such Blobs never get passed into
+	 * sqlMemCompare().
+	 */
+	assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
+	assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
+
+	if ((pB1->flags | pB2->flags) & MEM_Zero) {
+		if (pB1->flags & pB2->flags & MEM_Zero) {
+			return pB1->u.nZero - pB2->u.nZero;
+		} else if (pB1->flags & MEM_Zero) {
+			if (!isAllZero(pB2->z, pB2->n))
+				return -1;
+			return pB1->u.nZero - n2;
+		} else {
+			if (!isAllZero(pB1->z, pB1->n))
+				return +1;
+			return n1 - pB2->u.nZero;
+		}
+	}
+	c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
+	if (c)
+		return c;
+	return n1 - n2;
+}
+
+/*
+ * Compare the values contained by the two memory cells, returning
+ * negative, zero or positive if pMem1 is less than, equal to, or greater
+ * than pMem2. Sorting order is NULL's first, followed by numbers (integers
+ * and reals) sorted numerically, followed by text ordered by the collating
+ * sequence pColl and finally blob's ordered by memcmp().
+ *
+ * Two NULL values are considered equal by this function.
+ */
+int
+sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
+{
+	int f1, f2;
+	int combined_flags;
+
+	f1 = pMem1->flags;
+	f2 = pMem2->flags;
+	combined_flags = f1 | f2;
+
+	/* If one value is NULL, it is less than the other. If both values
+	 * are NULL, return 0.
+	 */
+	if (combined_flags & MEM_Null) {
+		return (f2 & MEM_Null) - (f1 & MEM_Null);
+	}
+
+	if ((combined_flags & MEM_Bool) != 0) {
+		if ((f1 & f2 & MEM_Bool) != 0) {
+			if (pMem1->u.b == pMem2->u.b)
+				return 0;
+			if (pMem1->u.b)
+				return 1;
+			return -1;
+		}
+		if ((f2 & MEM_Bool) != 0)
+			return +1;
+		return -1;
+	}
+
+	/* At least one of the two values is a number
+	 */
+	if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
+		if ((f1 & f2 & MEM_Int) != 0) {
+			if (pMem1->u.i < pMem2->u.i)
+				return -1;
+			if (pMem1->u.i > pMem2->u.i)
+				return +1;
+			return 0;
+		}
+		if ((f1 & f2 & MEM_UInt) != 0) {
+			if (pMem1->u.u < pMem2->u.u)
+				return -1;
+			if (pMem1->u.u > pMem2->u.u)
+				return +1;
+			return 0;
+		}
+		if ((f1 & f2 & MEM_Real) != 0) {
+			if (pMem1->u.r < pMem2->u.r)
+				return -1;
+			if (pMem1->u.r > pMem2->u.r)
+				return +1;
+			return 0;
+		}
+		if ((f1 & MEM_Int) != 0) {
+			if ((f2 & MEM_Real) != 0) {
+				return double_compare_nint64(pMem2->u.r,
+							     pMem1->u.i, -1);
+			} else {
+				return -1;
+			}
+		}
+		if ((f1 & MEM_UInt) != 0) {
+			if ((f2 & MEM_Real) != 0) {
+				return double_compare_uint64(pMem2->u.r,
+							     pMem1->u.u, -1);
+			} else if ((f2 & MEM_Int) != 0) {
+				return +1;
+			} else {
+				return -1;
+			}
+		}
+		if ((f1 & MEM_Real) != 0) {
+			if ((f2 & MEM_Int) != 0) {
+				return double_compare_nint64(pMem1->u.r,
+							     pMem2->u.i, 1);
+			} else if ((f2 & MEM_UInt) != 0) {
+				return double_compare_uint64(pMem1->u.r,
+							     pMem2->u.u, 1);
+			} else {
+				return -1;
+			}
+		}
+		return +1;
+	}
+
+	/* If one value is a string and the other is a blob, the string is less.
+	 * If both are strings, compare using the collating functions.
+	 */
+	if (combined_flags & MEM_Str) {
+		if ((f1 & MEM_Str) == 0) {
+			return 1;
+		}
+		if ((f2 & MEM_Str) == 0) {
+			return -1;
+		}
+		/* The collation sequence must be defined at this point, even if
+		 * the user deletes the collation sequence after the vdbe program is
+		 * compiled (this was not always the case).
+		 */
+		if (pColl) {
+			return vdbeCompareMemString(pMem1, pMem2, pColl);
+		} else {
+			size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
+			int res;
+			res = memcmp(pMem1->z, pMem2->z, n);
+			if (res == 0)
+				res = (int)pMem1->n - (int)pMem2->n;
+			return res;
+		}
+		/* If a NULL pointer was passed as the collate function, fall through
+		 * to the blob case and use memcmp().
+		 */
+	}
+
+	/* Both values must be blobs.  Compare using memcmp().  */
+	return sqlBlobCompare(pMem1, pMem2);
+}
+
+bool
+mem_is_type_compatible(struct Mem *mem, enum field_type type)
+{
+	enum mp_type mp_type = mem_mp_type(mem);
+	assert(mp_type < MP_EXT);
+	return field_mp_plain_type_is_compatible(type, mp_type, true);
+}
+
+/* Allocate memory for internal VDBE structure on region. */
+int
+vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
+{
+	vdbe_mem->n = size;
+	vdbe_mem->z = region_alloc(&fiber()->gc, size);
+	if (vdbe_mem->z == NULL)
+		return -1;
+	vdbe_mem->flags = MEM_Ephem | MEM_Blob;
+	assert(sqlVdbeCheckMemInvariants(vdbe_mem));
+	return 0;
+}
+
+/*
+ * Make a full copy of pFrom into pTo.  Prior contents of pTo are
+ * freed before the copy is made.
+ */
+int
+sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
+{
+	int rc = 0;
+
+	if (VdbeMemDynamic(pTo))
+		vdbeMemClearExternAndSetNull(pTo);
+	memcpy(pTo, pFrom, MEMCELLSIZE);
+	pTo->flags &= ~MEM_Dyn;
+	if (pTo->flags & (MEM_Str | MEM_Blob)) {
+		if (0 == (pFrom->flags & MEM_Static)) {
+			pTo->flags |= MEM_Ephem;
+			rc = sqlVdbeMemMakeWriteable(pTo);
+		}
+	}
+
+	return rc;
+}
+
+void
+sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
+{
+	assert(pTo->db == pFrom->db);
+	if (VdbeMemDynamic(pTo)) {
+		vdbeClrCopy(pTo, pFrom, srcType);
+		return;
+	}
+	memcpy(pTo, pFrom, MEMCELLSIZE);
+	if ((pFrom->flags & MEM_Static) == 0) {
+		pTo->flags &= ~(MEM_Dyn | MEM_Static | MEM_Ephem);
+		assert(srcType == MEM_Ephem || srcType == MEM_Static);
+		pTo->flags |= srcType;
+	}
+}
+
+/*
+ * Transfer the contents of pFrom to pTo. Any existing value in pTo is
+ * freed. If pFrom contains ephemeral data, a copy is made.
+ *
+ * pFrom contains an SQL NULL when this routine returns.
+ */
+void
+sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
+{
+	assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
+
+	sqlVdbeMemRelease(pTo);
+	memcpy(pTo, pFrom, sizeof(Mem));
+	pFrom->flags = MEM_Null;
+	pFrom->szMalloc = 0;
+}
+
+/*
+ * Change pMem so that its MEM_Str or MEM_Blob value is stored in
+ * MEM.zMalloc, where it can be safely written.
+ *
+ * Return 0 on success or -1 if malloc fails.
+ */
+int
+sqlVdbeMemMakeWriteable(Mem * pMem)
+{
+	if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
+		if (ExpandBlob(pMem))
+			return -1;
+		if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
+			if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
+				return -1;
+			}
+			pMem->z[pMem->n] = 0;
+			pMem->z[pMem->n + 1] = 0;
+			pMem->flags |= MEM_Term;
+		}
+	}
+	pMem->flags &= ~MEM_Ephem;
+#ifdef SQL_DEBUG
+	pMem->pScopyFrom = 0;
+#endif
+
+	return 0;
+}
+
+/*
+ * Release any memory resources held by the Mem.  Both the memory that is
+ * free by Mem.xDel and the Mem.zMalloc allocation are freed.
+ *
+ * Use this routine prior to clean up prior to abandoning a Mem, or to
+ * reset a Mem back to its minimum memory utilization.
+ *
+ * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
+ * prior to inserting new content into the Mem.
+ */
+void
+sqlVdbeMemRelease(Mem * p)
+{
+	assert(sqlVdbeCheckMemInvariants(p));
+	if (VdbeMemDynamic(p) || p->szMalloc) {
+		vdbeMemClear(p);
+	}
+}
+
+int
+sql_vdbemem_finalize(struct Mem *mem, struct func *func)
+{
+	assert(func != NULL);
+	assert(func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
+	assert(func->def->aggregate == FUNC_AGGREGATE_GROUP);
+	assert((mem->flags & MEM_Null) != 0 || func == mem->u.func);
+	sql_context ctx;
+	memset(&ctx, 0, sizeof(ctx));
+	Mem t;
+	memset(&t, 0, sizeof(t));
+	t.flags = MEM_Null;
+	t.db = mem->db;
+	t.field_type = field_type_MAX;
+	ctx.pOut = &t;
+	ctx.pMem = mem;
+	ctx.func = func;
+	((struct func_sql_builtin *)func)->finalize(&ctx);
+	assert((mem->flags & MEM_Dyn) == 0);
+	if (mem->szMalloc > 0)
+		sqlDbFree(mem->db, mem->zMalloc);
+	memcpy(mem, &t, sizeof(t));
+	return ctx.is_aborted ? -1 : 0;
+}
+
+int
+sqlVdbeCompareMsgpack(const char **key1,
+			  struct UnpackedRecord *unpacked, int key2_idx)
+{
+	const char *aKey1 = *key1;
+	Mem *pKey2 = unpacked->aMem + key2_idx;
+	Mem mem1;
+	int rc = 0;
+	switch (mp_typeof(*aKey1)) {
+	default:{
+			/* FIXME */
+			rc = -1;
+			break;
+		}
+	case MP_NIL:{
+			rc = -((pKey2->flags & MEM_Null) == 0);
+			mp_decode_nil(&aKey1);
+			break;
+		}
+	case MP_BOOL:{
+			mem1.u.b = mp_decode_bool(&aKey1);
+			if ((pKey2->flags & MEM_Bool) != 0) {
+				if (mem1.u.b != pKey2->u.b)
+					rc = mem1.u.b ? 1 : -1;
+			} else {
+				rc = (pKey2->flags & MEM_Null) != 0 ? 1 : -1;
+			}
+			break;
+		}
+	case MP_UINT:{
+			mem1.u.u = mp_decode_uint(&aKey1);
+			if ((pKey2->flags & MEM_Int) != 0) {
+				rc = +1;
+			} else if ((pKey2->flags & MEM_UInt) != 0) {
+				if (mem1.u.u < pKey2->u.u)
+					rc = -1;
+				else if (mem1.u.u > pKey2->u.u)
+					rc = +1;
+			} else if ((pKey2->flags & MEM_Real) != 0) {
+				rc = double_compare_uint64(pKey2->u.r,
+							   mem1.u.u, -1);
+			} else if ((pKey2->flags & MEM_Null) != 0) {
+				rc = 1;
+			} else if ((pKey2->flags & MEM_Bool) != 0) {
+				rc = 1;
+			} else {
+				rc = -1;
+			}
+			break;
+		}
+	case MP_INT:{
+			mem1.u.i = mp_decode_int(&aKey1);
+			if ((pKey2->flags & MEM_UInt) != 0) {
+				rc = -1;
+			} else if ((pKey2->flags & MEM_Int) != 0) {
+				if (mem1.u.i < pKey2->u.i) {
+					rc = -1;
+				} else if (mem1.u.i > pKey2->u.i) {
+					rc = +1;
+				}
+			} else if (pKey2->flags & MEM_Real) {
+				rc = double_compare_nint64(pKey2->u.r, mem1.u.i,
+							   -1);
+			} else if ((pKey2->flags & MEM_Null) != 0) {
+				rc = 1;
+			} else if ((pKey2->flags & MEM_Bool) != 0) {
+				rc = 1;
+			} else {
+				rc = -1;
+			}
+			break;
+		}
+	case MP_FLOAT:{
+			mem1.u.r = mp_decode_float(&aKey1);
+			goto do_float;
+		}
+	case MP_DOUBLE:{
+			mem1.u.r = mp_decode_double(&aKey1);
+ do_float:
+			if ((pKey2->flags & MEM_Int) != 0) {
+				rc = double_compare_nint64(mem1.u.r, pKey2->u.i,
+							   1);
+			} else if (pKey2->flags & MEM_UInt) {
+				rc = double_compare_uint64(mem1.u.r,
+							   pKey2->u.u, 1);
+			} else if (pKey2->flags & MEM_Real) {
+				if (mem1.u.r < pKey2->u.r) {
+					rc = -1;
+				} else if (mem1.u.r > pKey2->u.r) {
+					rc = +1;
+				}
+			} else if ((pKey2->flags & MEM_Null) != 0) {
+				rc = 1;
+			} else if ((pKey2->flags & MEM_Bool) != 0) {
+				rc = 1;
+			} else {
+				rc = -1;
+			}
+			break;
+		}
+	case MP_STR:{
+			if (pKey2->flags & MEM_Str) {
+				struct key_def *key_def = unpacked->key_def;
+				mem1.n = mp_decode_strl(&aKey1);
+				mem1.z = (char *)aKey1;
+				aKey1 += mem1.n;
+				struct coll *coll =
+					key_def->parts[key2_idx].coll;
+				if (coll != NULL) {
+					mem1.flags = MEM_Str;
+					rc = vdbeCompareMemString(&mem1, pKey2,
+								  coll);
+				} else {
+					goto do_bin_cmp;
+				}
+			} else {
+				rc = (pKey2->flags & MEM_Blob) ? -1 : +1;
+			}
+			break;
+		}
+	case MP_BIN:{
+			mem1.n = mp_decode_binl(&aKey1);
+			mem1.z = (char *)aKey1;
+			aKey1 += mem1.n;
+ do_blob:
+			if (pKey2->flags & MEM_Blob) {
+				if (pKey2->flags & MEM_Zero) {
+					if (!isAllZero
+					    ((const char *)mem1.z, mem1.n)) {
+						rc = 1;
+					} else {
+						rc = mem1.n - pKey2->u.nZero;
+					}
+				} else {
+					int nCmp;
+ do_bin_cmp:
+					nCmp = MIN(mem1.n, pKey2->n);
+					rc = memcmp(mem1.z, pKey2->z, nCmp);
+					if (rc == 0)
+						rc = mem1.n - pKey2->n;
+				}
+			} else {
+				rc = 1;
+			}
+			break;
+		}
+	case MP_ARRAY:
+	case MP_MAP:
+	case MP_EXT:{
+			mem1.z = (char *)aKey1;
+			mp_next(&aKey1);
+			mem1.n = aKey1 - (char *)mem1.z;
+			goto do_blob;
+		}
+	}
+	*key1 = aKey1;
+	return rc;
+}
+
+int
+sqlVdbeRecordCompareMsgpack(const void *key1,
+				struct UnpackedRecord *key2)
+{
+	int rc = 0;
+	u32 i, n = mp_decode_array((const char**)&key1);
+
+	n = MIN(n, key2->nField);
+
+	for (i = 0; i != n; i++) {
+		rc = sqlVdbeCompareMsgpack((const char**)&key1, key2, i);
+		if (rc != 0) {
+			if (key2->key_def->parts[i].sort_order !=
+			    SORT_ORDER_ASC) {
+				rc = -rc;
+			}
+			return rc;
+		}
+	}
+
+	key2->eqSeen = 1;
+	return key2->default_rc;
+}
+
+int
+vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
+{
+	const char *start_buf = buf;
+	switch (mp_typeof(*buf)) {
+	case MP_ARRAY: {
+		mem->z = (char *)buf;
+		mp_next(&buf);
+		mem->n = buf - mem->z;
+		mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+		mem->subtype = SQL_SUBTYPE_MSGPACK;
+		mem->field_type = FIELD_TYPE_ARRAY;
+		break;
+	}
+	case MP_MAP: {
+		mem->z = (char *)buf;
+		mp_next(&buf);
+		mem->n = buf - mem->z;
+		mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
+		mem->subtype = SQL_SUBTYPE_MSGPACK;
+		mem->field_type = FIELD_TYPE_MAP;
+		break;
+	}
+	case MP_EXT: {
+		mem->z = (char *)buf;
+		mp_next(&buf);
+		mem->n = buf - mem->z;
+		mem->flags = MEM_Blob | MEM_Ephem;
+		mem->field_type = FIELD_TYPE_VARBINARY;
+		break;
+	}
+	case MP_NIL: {
+		mp_decode_nil(&buf);
+		mem->flags = MEM_Null;
+		mem->field_type = field_type_MAX;
+		break;
+	}
+	case MP_BOOL: {
+		mem->u.b = mp_decode_bool(&buf);
+		mem->flags = MEM_Bool;
+		mem->field_type = FIELD_TYPE_BOOLEAN;
+		break;
+	}
+	case MP_UINT: {
+		uint64_t v = mp_decode_uint(&buf);
+		mem->u.u = v;
+		mem->flags = MEM_UInt;
+		mem->field_type = FIELD_TYPE_INTEGER;
+		break;
+	}
+	case MP_INT: {
+		mem->u.i = mp_decode_int(&buf);
+		mem->flags = MEM_Int;
+		mem->field_type = FIELD_TYPE_INTEGER;
+		break;
+	}
+	case MP_STR: {
+		/* XXX u32->int */
+		mem->n = (int) mp_decode_strl(&buf);
+		mem->flags = MEM_Str | MEM_Ephem;
+		mem->field_type = FIELD_TYPE_STRING;
+install_blob:
+		mem->z = (char *)buf;
+		buf += mem->n;
+		break;
+	}
+	case MP_BIN: {
+		/* XXX u32->int */
+		mem->n = (int) mp_decode_binl(&buf);
+		mem->flags = MEM_Blob | MEM_Ephem;
+		mem->field_type = FIELD_TYPE_VARBINARY;
+		goto install_blob;
+	}
+	case MP_FLOAT: {
+		mem->u.r = mp_decode_float(&buf);
+		if (sqlIsNaN(mem->u.r)) {
+			mem->flags = MEM_Null;
+			mem->field_type = field_type_MAX;
+		} else {
+			mem->flags = MEM_Real;
+			mem->field_type = FIELD_TYPE_DOUBLE;
+		}
+		break;
+	}
+	case MP_DOUBLE: {
+		mem->u.r = mp_decode_double(&buf);
+		if (sqlIsNaN(mem->u.r)) {
+			mem->flags = MEM_Null;
+			mem->field_type = field_type_MAX;
+		} else {
+			mem->flags = MEM_Real;
+			mem->field_type = FIELD_TYPE_DOUBLE;
+		}
+		break;
+	}
+	default:
+		unreachable();
+	}
+	*len = (uint32_t)(buf - start_buf);
+	return 0;
+}
+
+void
+mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
+{
+	assert(memIsValid(var));
+	int64_t i;
+	if (var->flags & MEM_Null) {
+		mpstream_encode_nil(stream);
+	} else if (var->flags & MEM_Real) {
+		mpstream_encode_double(stream, var->u.r);
+	} else if (var->flags & MEM_Int) {
+		i = var->u.i;
+		mpstream_encode_int(stream, i);
+	} else if (var->flags & MEM_UInt) {
+		i = var->u.u;
+		mpstream_encode_uint(stream, i);
+	} else if (var->flags & MEM_Str) {
+		mpstream_encode_strn(stream, var->z, var->n);
+	} else if (var->flags & MEM_Bool) {
+		mpstream_encode_bool(stream, var->u.b);
+	} else {
+		/*
+		 * Emit BIN header iff the BLOB doesn't store
+		 * MsgPack content.
+		 */
+		if (!mem_has_msgpack_subtype(var)) {
+			uint32_t binl = var->n +
+					((var->flags & MEM_Zero) ?
+					var->u.nZero : 0);
+			mpstream_encode_binl(stream, binl);
+		}
+		mpstream_memcpy(stream, var->z, var->n);
+		if (var->flags & MEM_Zero)
+			mpstream_memset(stream, 0, var->u.nZero);
+	}
+}
+
+char *
+sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
+			  uint32_t *tuple_size, struct region *region)
+{
+	size_t used = region_used(region);
+	bool is_error = false;
+	struct mpstream stream;
+	mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
+		      set_encode_error, &is_error);
+	mpstream_encode_array(&stream, field_count);
+	for (struct Mem *field = fields; field < fields + field_count; field++)
+		mpstream_encode_vdbe_mem(&stream, field);
+	mpstream_flush(&stream);
+	if (is_error) {
+		diag_set(OutOfMemory, stream.pos - stream.buf,
+			 "mpstream_flush", "stream");
+		return NULL;
+	}
+	*tuple_size = region_used(region) - used;
+	char *tuple = region_join(region, *tuple_size);
+	if (tuple == NULL) {
+		diag_set(OutOfMemory, *tuple_size, "region_join", "tuple");
+		return NULL;
+	}
+	mp_tuple_assert(tuple, tuple + *tuple_size);
+	return tuple;
+}
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
new file mode 100644
index 000000000..e78ebbe47
--- /dev/null
+++ b/src/box/sql/mem.h
@@ -0,0 +1,502 @@
+#pragma once
+/*
+ * Copyright 2010-2021, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ *    copyright notice, this list of conditions and the
+ *    following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ *    copyright notice, this list of conditions and the following
+ *    disclaimer in the documentation and/or other materials
+ *    provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#include "box/field_def.h"
+
+struct sql;
+struct Vdbe;
+struct region;
+struct mpstream;
+struct VdbeFrame;
+
+/*
+ * Internally, the vdbe manipulates nearly all SQL values as Mem
+ * structures. Each Mem struct may cache multiple representations (string,
+ * integer etc.) of the same value.
+ */
+struct Mem {
+	union MemValue {
+		double r;	/* Real value used when MEM_Real is set in flags */
+		i64 i;		/* Integer value used when MEM_Int is set in flags */
+		uint64_t u;	/* Unsigned integer used when MEM_UInt is set. */
+		bool b;         /* Boolean value used when MEM_Bool is set in flags */
+		int nZero;	/* Used when bit MEM_Zero is set in flags */
+		void *p;	/* Generic pointer */
+		/**
+		 * A pointer to function implementation.
+		 * Used only when flags==MEM_Agg.
+		 */
+		struct func *func;
+		struct VdbeFrame *pFrame;	/* Used when flags==MEM_Frame */
+	} u;
+	u32 flags;		/* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */
+	/** Subtype for this value. */
+	enum sql_subtype subtype;
+	/**
+	 * If value is fetched from tuple, then this property
+	 * contains type of corresponding space's field. If it's
+	 * value field_type_MAX then we can rely on on format
+	 * (msgpack) type which is represented by 'flags'.
+	 */
+	enum field_type field_type;
+	int n;			/* size (in bytes) of string value, excluding trailing '\0' */
+	char *z;		/* String or BLOB value */
+	/* ShallowCopy only needs to copy the information above */
+	char *zMalloc;		/* Space to hold MEM_Str or MEM_Blob if szMalloc>0 */
+	int szMalloc;		/* Size of the zMalloc allocation */
+	u32 uTemp;		/* Transient storage for serial_type in OP_MakeRecord */
+	sql *db;		/* The associated database connection */
+	void (*xDel) (void *);	/* Destructor for Mem.z - only valid if MEM_Dyn */
+#ifdef SQL_DEBUG
+	Mem *pScopyFrom;	/* This Mem is a shallow copy of pScopyFrom */
+	void *pFiller;		/* So that sizeof(Mem) is a multiple of 8 */
+#endif
+};
+
+/*
+ * Size of struct Mem not including the Mem.zMalloc member or anything that
+ * follows.
+ */
+#define MEMCELLSIZE offsetof(Mem,zMalloc)
+
+/* One or more of the following flags are set to indicate the validOK
+ * representations of the value stored in the Mem struct.
+ *
+ * If the MEM_Null flag is set, then the value is an SQL NULL value.
+ * No other flags may be set in this case.
+ *
+ * If the MEM_Str flag is set then Mem.z points at a string representation.
+ * Usually this is encoded in the same unicode encoding as the main
+ * database (see below for exceptions). If the MEM_Term flag is also
+ * set, then the string is nul terminated. The MEM_Int and MEM_Real
+ * flags may coexist with the MEM_Str flag.
+ */
+#define MEM_Null      0x0001	/* Value is NULL */
+#define MEM_Str       0x0002	/* Value is a string */
+#define MEM_Int       0x0004	/* Value is an integer */
+#define MEM_Real      0x0008	/* Value is a real number */
+#define MEM_Blob      0x0010	/* Value is a BLOB */
+#define MEM_Bool      0x0020    /* Value is a bool */
+#define MEM_UInt      0x0040	/* Value is an unsigned integer */
+#define MEM_Frame     0x0080	/* Value is a VdbeFrame object */
+#define MEM_Undefined 0x0100	/* Value is undefined */
+#define MEM_Cleared   0x0200	/* NULL set by OP_Null, not from data */
+#define MEM_TypeMask  0x83ff	/* Mask of type bits */
+
+/* Whenever Mem contains a valid string or blob representation, one of
+ * the following flags must be set to determine the memory management
+ * policy for Mem.z.  The MEM_Term flag tells us whether or not the
+ * string is \000 or \u0000 terminated
+ */
+#define MEM_Term      0x0400	/* String rep is nul terminated */
+#define MEM_Dyn       0x0800	/* Need to call Mem.xDel() on Mem.z */
+#define MEM_Static    0x1000	/* Mem.z points to a static string */
+#define MEM_Ephem     0x2000	/* Mem.z points to an ephemeral string */
+#define MEM_Agg       0x4000	/* Mem.z points to an agg function context */
+#define MEM_Zero      0x8000	/* Mem.i contains count of 0s appended to blob */
+#define MEM_Subtype   0x10000	/* Mem.eSubtype is valid */
+#define MEM_Ptr       0x20000	/* Value is a generic pointer */
+
+/**
+ * In contrast to Mem_TypeMask, this one allows to get
+ * pure type of memory cell, i.e. without _Dyn/_Zero and other
+ * auxiliary flags.
+ */
+enum {
+	MEM_PURE_TYPE_MASK = 0x7f
+};
+
+static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
+				     MEM_Blob | MEM_Bool | MEM_UInt),
+	      "value of type mask must consist of corresponding to memory "\
+	      "type bits");
+
+/**
+ * Simple type to str convertor. It is used to simplify
+ * error reporting.
+ */
+char *
+mem_type_to_str(const struct Mem *p);
+
+/*
+ * Return the MP_type of the value of the MEM.
+ * Analogue of sql_value_type() but operates directly on
+ * transparent memory cell.
+ */
+enum mp_type
+mem_mp_type(struct Mem *mem);
+
+enum mp_type
+sql_value_type(struct Mem *);
+u16
+numericType(Mem *pMem);
+
+int sqlValueBytes(struct Mem *);
+
+#ifdef SQL_DEBUG
+void sqlVdbeMemAboutToChange(struct Vdbe *, struct Mem *);
+int sqlVdbeCheckMemInvariants(struct Mem *);
+void sqlVdbePrintSql(Vdbe *);
+void sqlVdbeMemPrettyPrint(Mem * pMem, char *zBuf);
+void
+registerTrace(int iReg, Mem *p);
+
+/*
+ * Return true if a memory cell is not marked as invalid.  This macro
+ * is for use inside assert() statements only.
+ */
+#define memIsValid(M)  ((M)->flags & MEM_Undefined)==0
+#endif
+
+/*
+ * Invoke this macro on memory cells just prior to changing the
+ * value of the cell.  This macro verifies that shallow copies are
+ * not misused.  A shallow copy of a string or blob just copies a
+ * pointer to the string or blob, not the content.  If the original
+ * is changed while the copy is still in use, the string or blob might
+ * be changed out from under the copy.  This macro verifies that nothing
+ * like that ever happens.
+ */
+#ifdef SQL_DEBUG
+# define memAboutToChange(P,M) sqlVdbeMemAboutToChange(P,M)
+#else
+# define memAboutToChange(P,M)
+#endif
+
+/**
+ * Try to convert a string value into a numeric representation
+ * if we can do so without loss of information. Firstly, value
+ * is attempted to be converted to integer, and in case of fail -
+ * to floating point number. Note that function is assumed to be
+ * called on memory cell containing string, i.e. mem->type == MEM_Str.
+ *
+ * @param record Memory cell containing value to be converted.
+ * @retval 0 If value can be converted to integer or number.
+ * @retval -1 Otherwise.
+ */
+int
+mem_apply_numeric_type(struct Mem *record);
+int sqlVdbeMemRealify(struct Mem *);
+
+/**
+ * Convert @a mem to NUMBER type, so that after conversion it has
+ * one of types MEM_Real, MEM_Int or MEM_UInt. If conversion is
+ * not possible, function returns -1.
+ *
+ * Beware - this function changes value and type of @a mem
+ * argument.
+ */
+int
+vdbe_mem_numerify(struct Mem *mem);
+
+int sqlVdbeMemCast(struct Mem *, enum field_type type);
+int mem_apply_integer_type(struct Mem *);
+int sqlVdbeMemStringify(struct Mem *);
+int sqlVdbeMemNulTerminate(struct Mem *);
+int sqlVdbeMemExpandBlob(struct Mem *);
+#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
+void sql_value_apply_type(struct Mem *val, enum field_type type);
+
+
+/**
+ * Processing is determined by the field type parameter:
+ *
+ * INTEGER:
+ *    If memory holds floating point value and it can be
+ *    converted without loss (2.0 - > 2), it's type is
+ *    changed to INT. Otherwise, simply return success status.
+ *
+ * NUMBER:
+ *    If memory holds INT or floating point value,
+ *    no actions take place.
+ *
+ * STRING:
+ *    Convert mem to a string representation.
+ *
+ * SCALAR:
+ *    Mem is unchanged, but flag is set to BLOB in case of
+ *    scalar-like type. Otherwise, (MAP, ARRAY) conversion
+ *    is impossible.
+ *
+ * BOOLEAN:
+ *    If memory holds BOOLEAN no actions take place.
+ *
+ * ANY:
+ *    Mem is unchanged, no actions take place.
+ *
+ * MAP/ARRAY:
+ *    These types can't be casted to scalar ones, or to each
+ *    other. So the only valid conversion is to type itself.
+ *
+ * @param record The value to apply type to.
+ * @param type The type to be applied.
+ */
+int
+mem_apply_type(struct Mem *record, enum field_type type);
+
+/**
+ * Convert the numeric value contained in MEM to another numeric
+ * type.
+ *
+ * @param mem The MEM that contains the numeric value.
+ * @param type The type to convert to.
+ * @retval 0 if the conversion was successful, -1 otherwise.
+ */
+int
+mem_convert_to_numeric(struct Mem *mem, enum field_type type);
+
+/** Setters = Change MEM value. */
+
+int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
+int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
+
+void
+mem_set_bool(struct Mem *mem, bool value);
+
+/**
+ * Set VDBE memory register with given pointer as a data.
+ * @param mem VDBE memory register to update.
+ * @param ptr Pointer to use.
+ */
+void
+mem_set_ptr(struct Mem *mem, void *ptr);
+
+/**
+ * Set integer value. Depending on its sign MEM_Int (in case
+ * of negative value) or MEM_UInt flag is set.
+ */
+void
+mem_set_i64(struct Mem *mem, int64_t value);
+
+/** Set unsigned value and MEM_UInt flag. */
+void
+mem_set_u64(struct Mem *mem, uint64_t value);
+
+/**
+ * Set integer value. According to is_neg flag value is considered
+ * to be signed or unsigned.
+ */
+void
+mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
+
+/** Set double value and MEM_Real flag. */
+void
+mem_set_double(struct Mem *mem, double value);
+
+int
+sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
+void
+sqlVdbeMemInit(struct Mem *, sql *, u32);
+void
+sqlVdbeMemSetNull(struct Mem *);
+void
+sqlVdbeMemSetZeroBlob(struct Mem *, int);
+void sqlValueSetStr(struct Mem *, int, const void *,
+			void (*)(void *));
+void sqlValueSetNull(struct Mem *);
+void sqlValueFree(struct Mem *);
+struct Mem *sqlValueNew(struct sql *);
+
+/*
+ * Initialize an array of N Mem element.
+ */
+void
+initMemArray(Mem * p, int N, sql * db, u32 flags);
+
+/*
+ * Release an array of N Mem elements
+ */
+void
+releaseMemArray(Mem * p, int N);
+
+/*
+ * Clear any existing type flags from a Mem and replace them with f
+ */
+#define MemSetTypeFlag(p, f) \
+   ((p)->flags = ((p)->flags&~(MEM_TypeMask|MEM_Zero))|f)
+
+/** Getters. */
+
+int
+mem_value_bool(const struct Mem *mem, bool *b);
+int sqlVdbeIntValue(struct Mem *, int64_t *, bool *is_neg);
+int sqlVdbeRealValue(struct Mem *, double *);
+const void *
+sql_value_blob(struct Mem *);
+
+int
+sql_value_bytes(struct Mem *);
+
+double
+sql_value_double(struct Mem *);
+
+bool
+sql_value_boolean(struct Mem *val);
+
+int
+sql_value_int(struct Mem *);
+
+sql_int64
+sql_value_int64(struct Mem *);
+
+uint64_t
+sql_value_uint64(struct Mem *val);
+
+const unsigned char *
+sql_value_text(struct Mem *);
+
+const void *sqlValueText(struct Mem *);
+
+/**
+ * Return pointer to a string with the data type in the case of
+ * binary data stored in @a value. Otherwise, return the result
+ * of sql_value_text(). It is used due to the fact that not all
+ * binary strings can be displayed correctly (e.g. contain
+ * unprintable symbols).
+ */
+const char *
+sql_value_to_diag_str(struct Mem *value);
+#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
+
+enum sql_subtype
+sql_value_subtype(sql_value * pVal);
+
+const Mem *
+columnNullValue(void);
+
+/** Checkers. */
+
+static inline bool
+sql_value_is_null(struct Mem *value)
+{
+	return sql_value_type(value) == MP_NIL;
+}
+
+int sqlVdbeMemTooBig(Mem *);
+
+/* Return TRUE if Mem X contains dynamically allocated content - anything
+ * that needs to be deallocated to avoid a leak.
+ */
+#define VdbeMemDynamic(X)  \
+  (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
+
+
+int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
+
+/**
+ * Check that MEM_type of the mem is compatible with given type.
+ *
+ * @param mem The MEM that contains the value to check.
+ * @param type The type to check.
+ * @retval TRUE if the MEM_type of the value and the given type
+ *         are compatible, FALSE otherwise.
+ */
+bool
+mem_is_type_compatible(struct Mem *mem, enum field_type type);
+
+/** MEM manipulate functions. */
+
+int
+vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
+int sqlVdbeMemCopy(Mem *, const Mem *);
+void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
+void sqlVdbeMemMove(Mem *, Mem *);
+int sqlVdbeMemMakeWriteable(Mem *);
+void sqlVdbeMemRelease(Mem * p);
+
+/**
+ * Memory cell mem contains the context of an aggregate function.
+ * This routine calls the finalize method for that function. The
+ * result of the aggregate is stored back into mem.
+ *
+ * Returns -1 if the finalizer reports an error. 0 otherwise.
+ */
+int
+sql_vdbemem_finalize(struct Mem *mem, struct func *func);
+
+/** MEM and msgpack functions. */
+
+/**
+ * Perform comparison of two keys: one is packed and one is not.
+ *
+ * @param key1 Pointer to pointer to first key.
+ * @param unpacked Pointer to unpacked tuple.
+ * @param key2_idx index of key in umpacked record to compare.
+ *
+ * @retval +1 if key1 > pUnpacked[iKey2], -1 ptherwise.
+ */
+int sqlVdbeCompareMsgpack(const char **key1,
+			      struct UnpackedRecord *unpacked, int key2_idx);
+
+/**
+ * Perform comparison of two tuples: unpacked (key1) and packed (key2)
+ *
+ * @param key1 Packed key.
+ * @param unpacked Unpacked key.
+ *
+ * @retval +1 if key1 > unpacked, -1 otherwise.
+ */
+int sqlVdbeRecordCompareMsgpack(const void *key1,
+				    struct UnpackedRecord *key2);
+
+/**
+ * Decode msgpack and save value into VDBE memory cell.
+ *
+ * @param buf Buffer to deserialize msgpack from.
+ * @param mem Memory cell to write value into.
+ * @param len[out] Length of decoded part.
+ * @retval Return code: < 0 in case of error.
+ * @retval 0 on success.
+ */
+int
+vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len);
+
+/**
+ * Perform encoding memory variable to stream.
+ * @param stream Initialized mpstream encoder object.
+ * @param var Vdbe memory variable to encode with stream.
+ */
+void
+mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var);
+
+/**
+ * Perform encoding field_count Vdbe memory fields on region as
+ * msgpack array.
+ * @param fields The first Vdbe memory field to encode.
+ * @param field_count Count of fields to encode.
+ * @param[out] tuple_size Size of encoded tuple.
+ * @param region Region to use.
+ * @retval NULL on error, diag message is set.
+ * @retval Pointer to valid tuple on success.
+ */
+char *
+sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
+			  uint32_t *tuple_size, struct region *region);
diff --git a/src/box/sql/pragma.c b/src/box/sql/pragma.c
index c15f2e0d1..b820b492e 100644
--- a/src/box/sql/pragma.c
+++ b/src/box/sql/pragma.c
@@ -39,6 +39,7 @@
 #include "box/coll_id_cache.h"
 #include "sqlInt.h"
 #include "tarantoolInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "box/schema.h"
 #include "box/session.h"
diff --git a/src/box/sql/printf.c b/src/box/sql/printf.c
index a3ff5bb09..cf32ba3f3 100644
--- a/src/box/sql/printf.c
+++ b/src/box/sql/printf.c
@@ -10,6 +10,7 @@
  * sql.
  */
 #include "sqlInt.h"
+#include "mem.h"
 
 /*
  * Conversion types fall into various categories as defined by the
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index fb2795103..0fa388ae9 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -36,6 +36,7 @@
 #include "coll/coll.h"
 #include "sqlInt.h"
 #include "tarantoolInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "box/box.h"
 #include "box/coll_id_cache.h"
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 95a6fedc9..c1a42fc2f 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -351,20 +351,6 @@ sql_stricmp(const char *, const char *);
 int
 sql_strnicmp(const char *, const char *, int);
 
- const void *
-sql_value_blob(sql_value *);
-
-int
-sql_value_bytes(sql_value *);
-
-double
-sql_value_double(sql_value *);
-
-bool
-sql_value_boolean(sql_value *val);
-
-int
-sql_value_int(sql_value *);
 
 /**
  * Get row column subtype.
@@ -375,34 +361,6 @@ sql_value_int(sql_value *);
 enum sql_subtype
 sql_column_subtype(struct sql_stmt *stmt, int i);
 
-sql_int64
-sql_value_int64(sql_value *);
-
-uint64_t
-sql_value_uint64(sql_value *val);
-
-const unsigned char *
-sql_value_text(sql_value *);
-
-/**
- * Return pointer to a string with the data type in the case of
- * binary data stored in @a value. Otherwise, return the result
- * of sql_value_text(). It is used due to the fact that not all
- * binary strings can be displayed correctly (e.g. contain
- * unprintable symbols).
- */
-const char *
-sql_value_to_diag_str(sql_value *value);
-
-enum mp_type
-sql_value_type(sql_value *);
-
-static inline bool
-sql_value_is_null(sql_value *value)
-{
-	return sql_value_type(value) == MP_NIL;
-}
-
 sql *
 sql_context_db_handle(sql_context *);
 
@@ -4000,16 +3958,8 @@ int
 sql_rem_int(int64_t lhs, bool is_lhs_neg, int64_t rhs, bool is_rhs_neg,
 	    int64_t *res, bool *is_res_neg);
 
-const void *sqlValueText(sql_value *);
-int sqlValueBytes(sql_value *);
-void sqlValueSetStr(sql_value *, int, const void *,
-			void (*)(void *));
-void sqlValueSetNull(sql_value *);
-void sqlValueFree(sql_value *);
-sql_value *sqlValueNew(sql *);
 int sqlValueFromExpr(sql *, Expr *, enum field_type type,
 			 sql_value **);
-void sql_value_apply_type(sql_value *val, enum field_type type);
 
 extern const unsigned char sqlOpcodeProperty[];
 extern const unsigned char sqlUpperToLower[];
diff --git a/src/box/sql/trigger.c b/src/box/sql/trigger.c
index 0c387bc3b..7c983fea5 100644
--- a/src/box/sql/trigger.c
+++ b/src/box/sql/trigger.c
@@ -37,6 +37,7 @@
 #include "box/schema.h"
 #include "sqlInt.h"
 #include "tarantoolInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 
 /* See comment in sqlInt.h */
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 18806b93f..21abdcbd5 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -46,6 +46,7 @@
 #include "box/tuple.h"
 #include "box/port.h"
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "tarantoolInt.h"
 
@@ -57,21 +58,6 @@
 #include "box/sequence.h"
 #include "box/session_settings.h"
 
-/*
- * Invoke this macro on memory cells just prior to changing the
- * value of the cell.  This macro verifies that shallow copies are
- * not misused.  A shallow copy of a string or blob just copies a
- * pointer to the string or blob, not the content.  If the original
- * is changed while the copy is still in use, the string or blob might
- * be changed out from under the copy.  This macro verifies that nothing
- * like that ever happens.
- */
-#ifdef SQL_DEBUG
-# define memAboutToChange(P,M) sqlVdbeMemAboutToChange(P,M)
-#else
-# define memAboutToChange(P,M)
-#endif
-
 /*
  * The following global variable is incremented every time a cursor
  * moves, either by the OP_SeekXX, OP_Next, or OP_Prev opcodes.  The test
@@ -257,424 +243,6 @@ allocateCursor(
 	return pCx;
 }
 
-int
-mem_apply_numeric_type(struct Mem *record)
-{
-	if ((record->flags & MEM_Str) == 0)
-		return -1;
-	int64_t integer_value;
-	bool is_neg;
-	if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
-		mem_set_int(record, integer_value, is_neg);
-		return 0;
-	}
-	double float_value;
-	if (sqlAtoF(record->z, &float_value, record->n) == 0)
-		return -1;
-	mem_set_double(record, float_value);
-	return 0;
-}
-
-/**
- * Processing is determined by the field type parameter:
- *
- * INTEGER:
- *    If memory holds floating point value and it can be
- *    converted without loss (2.0 - > 2), it's type is
- *    changed to INT. Otherwise, simply return success status.
- *
- * NUMBER:
- *    If memory holds INT or floating point value,
- *    no actions take place.
- *
- * STRING:
- *    Convert mem to a string representation.
- *
- * SCALAR:
- *    Mem is unchanged, but flag is set to BLOB in case of
- *    scalar-like type. Otherwise, (MAP, ARRAY) conversion
- *    is impossible.
- *
- * BOOLEAN:
- *    If memory holds BOOLEAN no actions take place.
- *
- * ANY:
- *    Mem is unchanged, no actions take place.
- *
- * MAP/ARRAY:
- *    These types can't be casted to scalar ones, or to each
- *    other. So the only valid conversion is to type itself.
- *
- * @param record The value to apply type to.
- * @param type The type to be applied.
- */
-static int
-mem_apply_type(struct Mem *record, enum field_type type)
-{
-	if ((record->flags & MEM_Null) != 0)
-		return 0;
-	assert(type < field_type_MAX);
-	switch (type) {
-	case FIELD_TYPE_INTEGER:
-	case FIELD_TYPE_UNSIGNED:
-		if ((record->flags & (MEM_Bool | MEM_Blob)) != 0)
-			return -1;
-		if ((record->flags & MEM_UInt) == MEM_UInt)
-			return 0;
-		if ((record->flags & MEM_Real) == MEM_Real) {
-			double d = record->u.r;
-			if (d >= 0) {
-				if (double_compare_uint64(d, UINT64_MAX,
-							  1) > 0)
-					return 0;
-				if ((double)(uint64_t)d == d)
-					mem_set_u64(record, (uint64_t)d);
-			} else {
-				if (double_compare_nint64(d, INT64_MIN, 1) < 0)
-					return 0;
-				if ((double)(int64_t)d == d)
-					mem_set_int(record, (int64_t)d, true);
-			}
-			return 0;
-		}
-		if ((record->flags & MEM_Str) != 0) {
-			bool is_neg;
-			int64_t i;
-			if (sql_atoi64(record->z, &i, &is_neg, record->n) != 0)
-				return -1;
-			mem_set_int(record, i, is_neg);
-		}
-		if ((record->flags & MEM_Int) == MEM_Int) {
-			if (type == FIELD_TYPE_UNSIGNED)
-				return -1;
-			return 0;
-		}
-		return 0;
-	case FIELD_TYPE_BOOLEAN:
-		if ((record->flags & MEM_Bool) == MEM_Bool)
-			return 0;
-		return -1;
-	case FIELD_TYPE_NUMBER:
-		if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
-			return 0;
-		return sqlVdbeMemRealify(record);
-	case FIELD_TYPE_DOUBLE:
-		if ((record->flags & MEM_Real) != 0)
-			return 0;
-		return sqlVdbeMemRealify(record);
-	case FIELD_TYPE_STRING:
-		/*
-		 * Only attempt the conversion to TEXT if there is
-		 * an integer or real representation (BLOB and
-		 * NULL do not get converted).
-		 */
-		if ((record->flags & MEM_Str) == 0 &&
-		    (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
-			sqlVdbeMemStringify(record);
-		record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
-		return 0;
-	case FIELD_TYPE_VARBINARY:
-		if ((record->flags & MEM_Blob) == 0)
-			return -1;
-		return 0;
-	case FIELD_TYPE_SCALAR:
-		/* Can't cast MAP and ARRAY to scalar types. */
-		if ((record->flags & MEM_Subtype) != 0 &&
-		    record->subtype == SQL_SUBTYPE_MSGPACK) {
-			assert(mp_typeof(*record->z) == MP_MAP ||
-			       mp_typeof(*record->z) == MP_ARRAY);
-			return -1;
-		}
-		return 0;
-	case FIELD_TYPE_MAP:
-		if ((record->flags & MEM_Subtype) != 0 &&
-		    record->subtype == SQL_SUBTYPE_MSGPACK &&
-		    mp_typeof(*record->z) == MP_MAP)
-			return 0;
-		return -1;
-	case FIELD_TYPE_ARRAY:
-		if ((record->flags & MEM_Subtype) != 0 &&
-		    record->subtype == SQL_SUBTYPE_MSGPACK &&
-		    mp_typeof(*record->z) == MP_ARRAY)
-			return 0;
-		return -1;
-	case FIELD_TYPE_ANY:
-		return 0;
-	default:
-		return -1;
-	}
-}
-
-/*
- * Exported version of mem_apply_type(). This one works on sql_value*,
- * not the internal Mem* type.
- */
-void
-sql_value_apply_type(
-	sql_value *pVal,
-	enum field_type type)
-{
-	mem_apply_type((Mem *) pVal, type);
-}
-
-/**
- * Check that MEM_type of the mem is compatible with given type.
- *
- * @param mem The MEM that contains the value to check.
- * @param type The type to check.
- * @retval TRUE if the MEM_type of the value and the given type
- *         are compatible, FALSE otherwise.
- */
-static bool
-mem_is_type_compatible(struct Mem *mem, enum field_type type)
-{
-	enum mp_type mp_type = mem_mp_type(mem);
-	assert(mp_type < MP_EXT);
-	return field_mp_plain_type_is_compatible(type, mp_type, true);
-}
-
-/**
- * Convert the numeric value contained in MEM to double.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_double(struct Mem *mem)
-{
-	if ((mem->flags & MEM_Real) != 0)
-		return 0;
-	if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
-		return -1;
-	double d;
-	if ((mem->flags & MEM_Int) != 0)
-		d = (double)mem->u.i;
-	else
-		d = (double)mem->u.u;
-	mem_set_double(mem, d);
-	return 0;
-}
-
-/**
- * Convert the numeric value contained in MEM to unsigned.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_unsigned(struct Mem *mem)
-{
-	if ((mem->flags & MEM_UInt) != 0)
-		return 0;
-	if ((mem->flags & MEM_Int) != 0)
-		return -1;
-	if ((mem->flags & MEM_Real) == 0)
-		return -1;
-	double d = mem->u.r;
-	if (d < 0.0 || d >= (double)UINT64_MAX)
-		return -1;
-	mem_set_u64(mem, (uint64_t) d);
-	return 0;
-}
-
-/**
- * Convert the numeric value contained in MEM to integer.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_integer(struct Mem *mem)
-{
-	if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
-		return 0;
-	if ((mem->flags & MEM_Real) == 0)
-		return -1;
-	double d = mem->u.r;
-	if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
-		return -1;
-	if (d < (double)INT64_MAX)
-		mem_set_int(mem, (int64_t) d, d < 0);
-	else
-		mem_set_int(mem, (uint64_t) d, false);
-	return 0;
-}
-
-/**
- * Convert the numeric value contained in MEM to another numeric
- * type.
- *
- * @param mem The MEM that contains the numeric value.
- * @param type The type to convert to.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_numeric(struct Mem *mem, enum field_type type)
-{
-	assert(mp_type_is_numeric(mem_mp_type(mem)) &&
-	       sql_type_is_numeric(type));
-	assert(type != FIELD_TYPE_NUMBER);
-	if (type == FIELD_TYPE_DOUBLE)
-		return mem_convert_to_double(mem);
-	if (type == FIELD_TYPE_UNSIGNED)
-		return mem_convert_to_unsigned(mem);
-	assert(type == FIELD_TYPE_INTEGER);
-	return mem_convert_to_integer(mem);
-}
-
-/*
- * pMem currently only holds a string type (or maybe a BLOB that we can
- * interpret as a string if we want to).  Compute its corresponding
- * numeric type, if has one.  Set the pMem->u.r and pMem->u.i fields
- * accordingly.
- */
-static u16 SQL_NOINLINE computeNumericType(Mem *pMem)
-{
-	assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
-	assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
-	if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
-		return 0;
-	bool is_neg;
-	if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
-		return is_neg ? MEM_Int : MEM_UInt;
-	return MEM_Real;
-}
-
-/*
- * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
- * none.
- *
- * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
- * But it does set pMem->u.r and pMem->u.i appropriately.
- */
-static u16 numericType(Mem *pMem)
-{
-	if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
-		return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
-	if (pMem->flags & (MEM_Str|MEM_Blob)) {
-		return computeNumericType(pMem);
-	}
-	return 0;
-}
-
-#ifdef SQL_DEBUG
-/*
- * Write a nice string representation of the contents of cell pMem
- * into buffer zBuf, length nBuf.
- */
-void
-sqlVdbeMemPrettyPrint(Mem *pMem, char *zBuf)
-{
-	char *zCsr = zBuf;
-	int f = pMem->flags;
-
-	if (f&MEM_Blob) {
-		int i;
-		char c;
-		if (f & MEM_Dyn) {
-			c = 'z';
-			assert((f & (MEM_Static|MEM_Ephem))==0);
-		} else if (f & MEM_Static) {
-			c = 't';
-			assert((f & (MEM_Dyn|MEM_Ephem))==0);
-		} else if (f & MEM_Ephem) {
-			c = 'e';
-			assert((f & (MEM_Static|MEM_Dyn))==0);
-		} else {
-			c = 's';
-		}
-
-		sql_snprintf(100, zCsr, "%c", c);
-		zCsr += sqlStrlen30(zCsr);
-		sql_snprintf(100, zCsr, "%d[", pMem->n);
-		zCsr += sqlStrlen30(zCsr);
-		for(i=0; i<16 && i<pMem->n; i++) {
-			sql_snprintf(100, zCsr, "%02X", ((int)pMem->z[i] & 0xFF));
-			zCsr += sqlStrlen30(zCsr);
-		}
-		for(i=0; i<16 && i<pMem->n; i++) {
-			char z = pMem->z[i];
-			if (z<32 || z>126) *zCsr++ = '.';
-			else *zCsr++ = z;
-		}
-		sql_snprintf(100, zCsr, "]%s", "(8)");
-		zCsr += sqlStrlen30(zCsr);
-		if (f & MEM_Zero) {
-			sql_snprintf(100, zCsr,"+%dz",pMem->u.nZero);
-			zCsr += sqlStrlen30(zCsr);
-		}
-		*zCsr = '\0';
-	} else if (f & MEM_Str) {
-		int j, k;
-		zBuf[0] = ' ';
-		if (f & MEM_Dyn) {
-			zBuf[1] = 'z';
-			assert((f & (MEM_Static|MEM_Ephem))==0);
-		} else if (f & MEM_Static) {
-			zBuf[1] = 't';
-			assert((f & (MEM_Dyn|MEM_Ephem))==0);
-		} else if (f & MEM_Ephem) {
-			zBuf[1] = 'e';
-			assert((f & (MEM_Static|MEM_Dyn))==0);
-		} else {
-			zBuf[1] = 's';
-		}
-		k = 2;
-		sql_snprintf(100, &zBuf[k], "%d", pMem->n);
-		k += sqlStrlen30(&zBuf[k]);
-		zBuf[k++] = '[';
-		for(j=0; j<15 && j<pMem->n; j++) {
-			u8 c = pMem->z[j];
-			if (c>=0x20 && c<0x7f) {
-				zBuf[k++] = c;
-			} else {
-				zBuf[k++] = '.';
-			}
-		}
-		zBuf[k++] = ']';
-		sql_snprintf(100,&zBuf[k],"(8)");
-		k += sqlStrlen30(&zBuf[k]);
-		zBuf[k++] = 0;
-	}
-}
-#endif
-
-#ifdef SQL_DEBUG
-/*
- * Print the value of a register for tracing purposes:
- */
-static void
-memTracePrint(Mem *p)
-{
-	if (p->flags & MEM_Undefined) {
-		printf(" undefined");
-	} else if (p->flags & MEM_Null) {
-		printf(" NULL");
-	} else if ((p->flags & (MEM_Int|MEM_Str))==(MEM_Int|MEM_Str)) {
-		printf(" si:%lld", p->u.i);
-	} else if (p->flags & MEM_Int) {
-		printf(" i:%lld", p->u.i);
-	} else if (p->flags & MEM_UInt) {
-		printf(" u:%"PRIu64"", p->u.u);
-	} else if (p->flags & MEM_Real) {
-		printf(" r:%g", p->u.r);
-	} else if (p->flags & MEM_Bool) {
-		printf(" bool:%s", SQL_TOKEN_BOOLEAN(p->u.b));
-	} else {
-		char zBuf[200];
-		sqlVdbeMemPrettyPrint(p, zBuf);
-		printf(" %s", zBuf);
-	}
-	if (p->flags & MEM_Subtype) printf(" subtype=0x%02x", p->subtype);
-}
-static void
-registerTrace(int iReg, Mem *p) {
-	printf("REG[%d] = ", iReg);
-	memTracePrint(p);
-	printf("\n");
-}
-#endif
-
 #ifdef SQL_DEBUG
 #  define REGISTER_TRACE(P,R,M)					\
 	if(P->sql_flags&SQL_VdbeTrace) registerTrace(R,M);
@@ -727,43 +295,6 @@ vdbe_add_new_autoinc_id(struct Vdbe *vdbe, int64_t id)
 	return 0;
 }
 
-char *
-mem_type_to_str(const struct Mem *p)
-{
-	assert(p != NULL);
-	switch (p->flags & MEM_PURE_TYPE_MASK) {
-	case MEM_Null:
-		return "NULL";
-	case MEM_Str:
-		return "text";
-	case MEM_Int:
-		return "integer";
-	case MEM_UInt:
-		return "unsigned";
-	case MEM_Real:
-		return "real";
-	case MEM_Blob:
-		return "varbinary";
-	case MEM_Bool:
-		return "boolean";
-	default:
-		unreachable();
-	}
-}
-
-/* Allocate memory for internal VDBE structure on region. */
-static int
-vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
-{
-	vdbe_mem->n = size;
-	vdbe_mem->z = region_alloc(&fiber()->gc, size);
-	if (vdbe_mem->z == NULL)
-		return -1;
-	vdbe_mem->flags = MEM_Ephem | MEM_Blob;
-	assert(sqlVdbeCheckMemInvariants(vdbe_mem));
-	return 0;
-}
-
 static inline const struct tuple_field *
 vdbe_field_ref_fetch_field(struct vdbe_field_ref *field_ref, uint32_t fieldno)
 {
diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h
index 48815d4ec..118f1cd83 100644
--- a/src/box/sql/vdbe.h
+++ b/src/box/sql/vdbe.h
@@ -268,7 +268,6 @@ void sqlVdbeSwap(Vdbe *, Vdbe *);
 VdbeOp *sqlVdbeTakeOpArray(Vdbe *, int *, int *);
 sql_value *sqlVdbeGetBoundValue(Vdbe *, int, u8);
 char *sqlVdbeExpandSql(Vdbe *, const char *);
-int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
 
 /**
  * Perform unpacking of provided message pack.
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 7205f1af3..089ac7575 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -155,151 +155,6 @@ struct VdbeFrame {
 	int nDbChange;		/* Value of db->nChange */
 };
 
-#define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
-
-/*
- * Internally, the vdbe manipulates nearly all SQL values as Mem
- * structures. Each Mem struct may cache multiple representations (string,
- * integer etc.) of the same value.
- */
-struct Mem {
-	union MemValue {
-		double r;	/* Real value used when MEM_Real is set in flags */
-		i64 i;		/* Integer value used when MEM_Int is set in flags */
-		uint64_t u;	/* Unsigned integer used when MEM_UInt is set. */
-		bool b;         /* Boolean value used when MEM_Bool is set in flags */
-		int nZero;	/* Used when bit MEM_Zero is set in flags */
-		void *p;	/* Generic pointer */
-		/**
-		 * A pointer to function implementation.
-		 * Used only when flags==MEM_Agg.
-		 */
-		struct func *func;
-		VdbeFrame *pFrame;	/* Used when flags==MEM_Frame */
-	} u;
-	u32 flags;		/* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */
-	/** Subtype for this value. */
-	enum sql_subtype subtype;
-	/**
-	 * If value is fetched from tuple, then this property
-	 * contains type of corresponding space's field. If it's
-	 * value field_type_MAX then we can rely on on format
-	 * (msgpack) type which is represented by 'flags'.
-	 */
-	enum field_type field_type;
-	int n;			/* size (in bytes) of string value, excluding trailing '\0' */
-	char *z;		/* String or BLOB value */
-	/* ShallowCopy only needs to copy the information above */
-	char *zMalloc;		/* Space to hold MEM_Str or MEM_Blob if szMalloc>0 */
-	int szMalloc;		/* Size of the zMalloc allocation */
-	u32 uTemp;		/* Transient storage for serial_type in OP_MakeRecord */
-	sql *db;		/* The associated database connection */
-	void (*xDel) (void *);	/* Destructor for Mem.z - only valid if MEM_Dyn */
-#ifdef SQL_DEBUG
-	Mem *pScopyFrom;	/* This Mem is a shallow copy of pScopyFrom */
-	void *pFiller;		/* So that sizeof(Mem) is a multiple of 8 */
-#endif
-};
-
-/*
- * Size of struct Mem not including the Mem.zMalloc member or anything that
- * follows.
- */
-#define MEMCELLSIZE offsetof(Mem,zMalloc)
-
-/* One or more of the following flags are set to indicate the validOK
- * representations of the value stored in the Mem struct.
- *
- * If the MEM_Null flag is set, then the value is an SQL NULL value.
- * No other flags may be set in this case.
- *
- * If the MEM_Str flag is set then Mem.z points at a string representation.
- * Usually this is encoded in the same unicode encoding as the main
- * database (see below for exceptions). If the MEM_Term flag is also
- * set, then the string is nul terminated. The MEM_Int and MEM_Real
- * flags may coexist with the MEM_Str flag.
- */
-#define MEM_Null      0x0001	/* Value is NULL */
-#define MEM_Str       0x0002	/* Value is a string */
-#define MEM_Int       0x0004	/* Value is an integer */
-#define MEM_Real      0x0008	/* Value is a real number */
-#define MEM_Blob      0x0010	/* Value is a BLOB */
-#define MEM_Bool      0x0020    /* Value is a bool */
-#define MEM_UInt      0x0040	/* Value is an unsigned integer */
-#define MEM_Frame     0x0080	/* Value is a VdbeFrame object */
-#define MEM_Undefined 0x0100	/* Value is undefined */
-#define MEM_Cleared   0x0200	/* NULL set by OP_Null, not from data */
-#define MEM_TypeMask  0x83ff	/* Mask of type bits */
-
-/* Whenever Mem contains a valid string or blob representation, one of
- * the following flags must be set to determine the memory management
- * policy for Mem.z.  The MEM_Term flag tells us whether or not the
- * string is \000 or \u0000 terminated
- */
-#define MEM_Term      0x0400	/* String rep is nul terminated */
-#define MEM_Dyn       0x0800	/* Need to call Mem.xDel() on Mem.z */
-#define MEM_Static    0x1000	/* Mem.z points to a static string */
-#define MEM_Ephem     0x2000	/* Mem.z points to an ephemeral string */
-#define MEM_Agg       0x4000	/* Mem.z points to an agg function context */
-#define MEM_Zero      0x8000	/* Mem.i contains count of 0s appended to blob */
-#define MEM_Subtype   0x10000	/* Mem.eSubtype is valid */
-#define MEM_Ptr       0x20000	/* Value is a generic pointer */
-
-/**
- * In contrast to Mem_TypeMask, this one allows to get
- * pure type of memory cell, i.e. without _Dyn/_Zero and other
- * auxiliary flags.
- */
-enum {
-	MEM_PURE_TYPE_MASK = 0x7f
-};
-
-static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
-				     MEM_Blob | MEM_Bool | MEM_UInt),
-	      "value of type mask must consist of corresponding to memory "\
-	      "type bits");
-
-/**
- * Simple type to str convertor. It is used to simplify
- * error reporting.
- */
-char *
-mem_type_to_str(const struct Mem *p);
-
-/**
- * Try to convert a string value into a numeric representation
- * if we can do so without loss of information. Firstly, value
- * is attempted to be converted to integer, and in case of fail -
- * to floating point number. Note that function is assumed to be
- * called on memory cell containing string, i.e. mem->type == MEM_Str.
- *
- * @param record Memory cell containing value to be converted.
- * @retval 0 If value can be converted to integer or number.
- * @retval -1 Otherwise.
- */
-int
-mem_apply_numeric_type(struct Mem *record);
-
-/* Return TRUE if Mem X contains dynamically allocated content - anything
- * that needs to be deallocated to avoid a leak.
- */
-#define VdbeMemDynamic(X)  \
-  (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
-
-/*
- * Clear any existing type flags from a Mem and replace them with f
- */
-#define MemSetTypeFlag(p, f) \
-   ((p)->flags = ((p)->flags&~(MEM_TypeMask|MEM_Zero))|f)
-
-/*
- * Return true if a memory cell is not marked as invalid.  This macro
- * is for use inside assert() statements only.
- */
-#ifdef SQL_DEBUG
-#define memIsValid(M)  ((M)->flags & MEM_Undefined)==0
-#endif
-
 /*
  * The "context" argument for an installable function.  A pointer to an
  * instance of this structure is the first argument to the routines used
@@ -477,83 +332,8 @@ int sqlVdbeExec(Vdbe *);
 int sqlVdbeList(Vdbe *);
 
 int sqlVdbeHalt(Vdbe *);
-int sqlVdbeMemTooBig(Mem *);
-int sqlVdbeMemCopy(Mem *, const Mem *);
-void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
-void sqlVdbeMemMove(Mem *, Mem *);
-int sqlVdbeMemNulTerminate(Mem *);
-int sqlVdbeMemSetStr(Mem *, const char *, int, u8, void (*)(void *));
-
-void
-mem_set_bool(struct Mem *mem, bool value);
-
-/**
- * Set VDBE memory register with given pointer as a data.
- * @param mem VDBE memory register to update.
- * @param ptr Pointer to use.
- */
-void
-mem_set_ptr(struct Mem *mem, void *ptr);
-
-/**
- * Set integer value. Depending on its sign MEM_Int (in case
- * of negative value) or MEM_UInt flag is set.
- */
-void
-mem_set_i64(struct Mem *mem, int64_t value);
-
-/** Set unsigned value and MEM_UInt flag. */
-void
-mem_set_u64(struct Mem *mem, uint64_t value);
-
-/**
- * Set integer value. According to is_neg flag value is considered
- * to be signed or unsigned.
- */
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
-
-/** Set double value and MEM_Real flag. */
-void
-mem_set_double(struct Mem *mem, double value);
 
-void sqlVdbeMemInit(Mem *, sql *, u32);
-void sqlVdbeMemSetNull(Mem *);
-void sqlVdbeMemSetZeroBlob(Mem *, int);
-int sqlVdbeMemMakeWriteable(Mem *);
-int sqlVdbeMemStringify(Mem *);
-int sqlVdbeIntValue(Mem *, int64_t *, bool *is_neg);
-
-int sqlVdbeRealValue(Mem *, double *);
-
-int
-mem_value_bool(const struct Mem *mem, bool *b);
-
-int mem_apply_integer_type(Mem *);
-int sqlVdbeMemRealify(Mem *);
-
-/**
- * Convert @a mem to NUMBER type, so that after conversion it has
- * one of types MEM_Real, MEM_Int or MEM_UInt. If conversion is
- * not possible, function returns -1.
- *
- * Beware - this function changes value and type of @a mem
- * argument.
- */
-int
-vdbe_mem_numerify(struct Mem *mem);
-
-int sqlVdbeMemCast(Mem *, enum field_type type);
 int sqlVdbeMemFromBtree(BtCursor *, u32, u32, Mem *);
-void sqlVdbeMemRelease(Mem * p);
-
-/*
- * Return the MP_type of the value of the MEM.
- * Analogue of sql_value_type() but operates directly on
- * transparent memory cell.
- */
-enum mp_type
-mem_mp_type(struct Mem *mem);
 
 /**
  * In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
@@ -566,19 +346,7 @@ mem_mp_type(struct Mem *mem);
 #define mp_type_is_numeric(X) ((X) == MP_INT || (X) == MP_UINT ||\
 			       (X) == MP_DOUBLE)
 
-/**
- * Memory cell mem contains the context of an aggregate function.
- * This routine calls the finalize method for that function. The
- * result of the aggregate is stored back into mem.
- *
- * Returns -1 if the finalizer reports an error. 0 otherwise.
- */
-int
-sql_vdbemem_finalize(struct Mem *mem, struct func *func);
-
 const char *sqlOpcodeName(int);
-int sqlVdbeMemGrow(Mem * pMem, int n, int preserve);
-int sqlVdbeMemClearAndResize(Mem * pMem, int n);
 int sqlVdbeCloseStatement(Vdbe *, int);
 void sqlVdbeFrameDelete(VdbeFrame *);
 int sqlVdbeFrameRestore(VdbeFrame *);
@@ -596,57 +364,12 @@ int sqlVdbeSorterRewind(const VdbeCursor *, int *);
 int sqlVdbeSorterWrite(const VdbeCursor *, Mem *);
 int sqlVdbeSorterCompare(const VdbeCursor *, Mem *, int, int *);
 
-#ifdef SQL_DEBUG
-void sqlVdbeMemAboutToChange(Vdbe *, Mem *);
-int sqlVdbeCheckMemInvariants(Mem *);
-#endif
-
 int sqlVdbeCheckFk(Vdbe *, int);
 
 int sqlVdbeMemTranslate(Mem *, u8);
-#ifdef SQL_DEBUG
-void sqlVdbePrintSql(Vdbe *);
-void sqlVdbeMemPrettyPrint(Mem * pMem, char *zBuf);
-#endif
 int sqlVdbeMemHandleBom(Mem * pMem);
 
-int sqlVdbeMemExpandBlob(Mem *);
-#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
-
-/**
- * Perform comparison of two keys: one is packed and one is not.
- *
- * @param key1 Pointer to pointer to first key.
- * @param unpacked Pointer to unpacked tuple.
- * @param key2_idx index of key in umpacked record to compare.
- *
- * @retval +1 if key1 > pUnpacked[iKey2], -1 ptherwise.
- */
-int sqlVdbeCompareMsgpack(const char **key1,
-			      struct UnpackedRecord *unpacked, int key2_idx);
-
-/**
- * Perform comparison of two tuples: unpacked (key1) and packed (key2)
- *
- * @param key1 Packed key.
- * @param unpacked Unpacked key.
- *
- * @retval +1 if key1 > unpacked, -1 otherwise.
- */
-int sqlVdbeRecordCompareMsgpack(const void *key1,
-				    struct UnpackedRecord *key2);
 
-/**
- * Decode msgpack and save value into VDBE memory cell.
- *
- * @param buf Buffer to deserialize msgpack from.
- * @param mem Memory cell to write value into.
- * @param len[out] Length of decoded part.
- * @retval Return code: < 0 in case of error.
- * @retval 0 on success.
- */
-int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len);
 
 struct mpstream;
 struct region;
@@ -658,26 +381,4 @@ set_encode_error(void *error_ctx)
 	*(bool *)error_ctx = true;
 }
 
-/**
- * Perform encoding memory variable to stream.
- * @param stream Initialized mpstream encoder object.
- * @param var Vdbe memory variable to encode with stream.
- */
-void
-mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var);
-
-/**
- * Perform encoding field_count Vdbe memory fields on region as
- * msgpack array.
- * @param fields The first Vdbe memory field to encode.
- * @param field_count Count of fields to encode.
- * @param[out] tuple_size Size of encoded tuple.
- * @param region Region to use.
- * @retval NULL on error, diag message is set.
- * @retval Pointer to valid tuple on success.
- */
-char *
-sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
-			  uint32_t *tuple_size, struct region *region);
-
 #endif				/* !defined(SQL_VDBEINT_H) */
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 7c59ef83f..24fa99f17 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -35,6 +35,7 @@
  * VDBE.
  */
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "box/session.h"
 
@@ -122,101 +123,6 @@ sql_metadata_is_full()
 	return current_session()->sql_flags & SQL_FullMetadata;
 }
 
-/**************************** sql_value_  ******************************
- * The following routines extract information from a Mem or sql_value
- * structure.
- */
-const void *
-sql_value_blob(sql_value * pVal)
-{
-	Mem *p = (Mem *) pVal;
-	if (p->flags & (MEM_Blob | MEM_Str)) {
-		if (ExpandBlob(p) != 0) {
-			assert(p->flags == MEM_Null && p->z == 0);
-			return 0;
-		}
-		p->flags |= MEM_Blob;
-		return p->n ? p->z : 0;
-	} else {
-		return sql_value_text(pVal);
-	}
-}
-
-int
-sql_value_bytes(sql_value * pVal)
-{
-	return sqlValueBytes(pVal);
-}
-
-double
-sql_value_double(sql_value * pVal)
-{
-	double v = 0.0;
-	sqlVdbeRealValue((Mem *) pVal, &v);
-	return v;
-}
-
-bool
-sql_value_boolean(sql_value *val)
-{
-	bool b = false;
-	int rc = mem_value_bool((struct Mem *) val, &b);
-	assert(rc == 0);
-	(void) rc;
-	return b;
-}
-
-int
-sql_value_int(sql_value * pVal)
-{
-	int64_t i = 0;
-	bool is_neg;
-	sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
-	return (int)i;
-}
-
-sql_int64
-sql_value_int64(sql_value * pVal)
-{
-	int64_t i = 0;
-	bool unused;
-	sqlVdbeIntValue((Mem *) pVal, &i, &unused);
-	return i;
-}
-
-uint64_t
-sql_value_uint64(sql_value *val)
-{
-	int64_t i = 0;
-	bool is_neg;
-	sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
-	assert(!is_neg);
-	return i;
-}
-
-enum sql_subtype
-sql_value_subtype(sql_value * pVal)
-{
-	return (pVal->flags & MEM_Subtype) != 0 ? pVal->subtype : SQL_SUBTYPE_NO;
-}
-
-const unsigned char *
-sql_value_text(sql_value * pVal)
-{
-	return (const unsigned char *)sqlValueText(pVal);
-}
-
-/* EVIDENCE-OF: R-12793-43283 Every value in sql has one of five
- * fundamental datatypes: 64-bit signed integer 64-bit IEEE floating
- * point number string BLOB NULL
- */
-enum mp_type
-sql_value_type(sql_value *pVal)
-{
-	struct Mem *mem = (struct Mem *) pVal;
-	return mem_mp_type(mem);
-}
-
 /* Make a copy of an sql_value object
  */
 sql_value *
@@ -582,47 +488,6 @@ sql_data_count(sql_stmt * pStmt)
 	return pVm->nResColumn;
 }
 
-/*
- * Return a pointer to static memory containing an SQL NULL value.
- */
-static const Mem *
-columnNullValue(void)
-{
-	/* Even though the Mem structure contains an element
-	 * of type i64, on certain architectures (x86) with certain compiler
-	 * switches (-Os), gcc may align this Mem object on a 4-byte boundary
-	 * instead of an 8-byte one. This all works fine, except that when
-	 * running with SQL_DEBUG defined the sql code sometimes assert()s
-	 * that a Mem structure is located on an 8-byte boundary. To prevent
-	 * these assert()s from failing, when building with SQL_DEBUG defined
-	 * using gcc, we force nullMem to be 8-byte aligned using the magical
-	 * __attribute__((aligned(8))) macro.
-	 */
-	static const Mem nullMem
-#if defined(SQL_DEBUG) && defined(__GNUC__)
-	    __attribute__ ((aligned(8)))
-#endif
-	    = {
-		/* .u          = */  {
-		0},
-		    /* .flags      = */ (u16) MEM_Null,
-		    /* .eSubtype   = */ (u8) 0,
-		    /* .field_type = */ field_type_MAX,
-		    /* .n          = */ (int)0,
-		    /* .z          = */ (char *)0,
-		    /* .zMalloc    = */ (char *)0,
-		    /* .szMalloc   = */ (int)0,
-		    /* .uTemp      = */ (u32) 0,
-		    /* .db         = */ (sql *) 0,
-		    /* .xDel       = */ (void (*)(void *))0,
-#ifdef SQL_DEBUG
-		    /* .pScopyFrom = */ (Mem *) 0,
-		    /* .pFiller    = */ (void *)0,
-#endif
-	};
-	return &nullMem;
-}
-
 /*
  * Check to see if column iCol of the given statement is valid.  If
  * it is, return a pointer to the Mem for the value of that column.
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 90c4b73c0..9153df1df 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -41,6 +41,7 @@
 #include "box/txn.h"
 #include "msgpuck/msgpuck.h"
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "tarantoolInt.h"
 #include "box/execute.h"
@@ -1194,65 +1195,6 @@ sqlVdbePrintOp(FILE * pOut, int pc, Op * pOp)
 }
 #endif
 
-/*
- * Initialize an array of N Mem element.
- */
-static void
-initMemArray(Mem * p, int N, sql * db, u32 flags)
-{
-	while ((N--) > 0) {
-		p->db = db;
-		p->flags = flags;
-		p->szMalloc = 0;
-		p->field_type = field_type_MAX;
-#ifdef SQL_DEBUG
-		p->pScopyFrom = 0;
-#endif
-		p++;
-	}
-}
-
-/*
- * Release an array of N Mem elements
- */
-static void
-releaseMemArray(Mem * p, int N)
-{
-	if (p && N) {
-		Mem *pEnd = &p[N];
-		sql *db = p->db;
-		do {
-			assert((&p[1]) == pEnd || p[0].db == p[1].db);
-			assert(sqlVdbeCheckMemInvariants(p));
-
-			/* This block is really an inlined version of sqlVdbeMemRelease()
-			 * that takes advantage of the fact that the memory cell value is
-			 * being set to NULL after releasing any dynamic resources.
-			 *
-			 * The justification for duplicating code is that according to
-			 * callgrind, this causes a certain test case to hit the CPU 4.7
-			 * percent less (x86 linux, gcc version 4.1.2, -O6) than if
-			 * sqlMemRelease() were called from here. With -O2, this jumps
-			 * to 6.6 percent. The test case is inserting 1000 rows into a table
-			 * with no indexes using a single prepared INSERT statement, bind()
-			 * and reset(). Inserts are grouped into a transaction.
-			 */
-			testcase(p->flags & MEM_Agg);
-			testcase(p->flags & MEM_Dyn);
-			testcase(p->flags & MEM_Frame);
-			if (p->
-			    flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
-				sqlVdbeMemRelease(p);
-			} else if (p->szMalloc) {
-				sqlDbFree(db, p->zMalloc);
-				p->szMalloc = 0;
-			}
-
-			p->flags = MEM_Undefined;
-		} while ((++p) < pEnd);
-	}
-}
-
 /*
  * Delete a VdbeFrame object and its contents. VdbeFrame objects are
  * allocated by the OP_Program opcode in sqlVdbeExec().
@@ -1447,30 +1389,6 @@ sqlVdbeList(Vdbe * p)
 	return rc;
 }
 
-#ifdef SQL_DEBUG
-/*
- * Print the SQL that was used to generate a VDBE program.
- */
-void
-sqlVdbePrintSql(Vdbe * p)
-{
-	const char *z = 0;
-	if (p->zSql) {
-		z = p->zSql;
-	} else if (p->nOp >= 1) {
-		const VdbeOp *pOp = &p->aOp[0];
-		if (pOp->opcode == OP_Init && pOp->p4.z != 0) {
-			z = pOp->p4.z;
-			while (sqlIsspace(*z))
-				z++;
-		}
-	}
-	if (z)
-		printf("SQL: [%s]\n", z);
-}
-#endif
-
-
 /* An instance of this object describes bulk memory available for use
  * by subcomponents of a prepared statement.  Space is allocated out
  * of a ReusableSpace object by the allocSpace() routine below.
@@ -2334,204 +2252,6 @@ sqlVdbeAllocUnpackedRecord(struct sql *db, struct key_def *key_def)
 	return p;
 }
 
-/*
- * Both *pMem1 and *pMem2 contain string values. Compare the two values
- * using the collation sequence pColl. As usual, return a negative , zero
- * or positive value if *pMem1 is less than, equal to or greater than
- * *pMem2, respectively. Similar in spirit to "rc = (*pMem1) - (*pMem2);".
- *
- * Strungs assume to be UTF-8 encoded
- */
-static int
-vdbeCompareMemString(const Mem * pMem1, const Mem * pMem2,
-		     const struct coll * pColl)
-{
-	return pColl->cmp(pMem1->z, (size_t)pMem1->n,
-			      pMem2->z, (size_t)pMem2->n, pColl);
-}
-
-/*
- * The input pBlob is guaranteed to be a Blob that is not marked
- * with MEM_Zero.  Return true if it could be a zero-blob.
- */
-static int
-isAllZero(const char *z, int n)
-{
-	int i;
-	for (i = 0; i < n; i++) {
-		if (z[i])
-			return 0;
-	}
-	return 1;
-}
-
-/*
- * Compare two blobs.  Return negative, zero, or positive if the first
- * is less than, equal to, or greater than the second, respectively.
- * If one blob is a prefix of the other, then the shorter is the lessor.
- */
-static SQL_NOINLINE int
-sqlBlobCompare(const Mem * pB1, const Mem * pB2)
-{
-	int c;
-	int n1 = pB1->n;
-	int n2 = pB2->n;
-
-	/* It is possible to have a Blob value that has some non-zero content
-	 * followed by zero content.  But that only comes up for Blobs formed
-	 * by the OP_MakeRecord opcode, and such Blobs never get passed into
-	 * sqlMemCompare().
-	 */
-	assert((pB1->flags & MEM_Zero) == 0 || n1 == 0);
-	assert((pB2->flags & MEM_Zero) == 0 || n2 == 0);
-
-	if ((pB1->flags | pB2->flags) & MEM_Zero) {
-		if (pB1->flags & pB2->flags & MEM_Zero) {
-			return pB1->u.nZero - pB2->u.nZero;
-		} else if (pB1->flags & MEM_Zero) {
-			if (!isAllZero(pB2->z, pB2->n))
-				return -1;
-			return pB1->u.nZero - n2;
-		} else {
-			if (!isAllZero(pB1->z, pB1->n))
-				return +1;
-			return n1 - pB2->u.nZero;
-		}
-	}
-	c = memcmp(pB1->z, pB2->z, n1 > n2 ? n2 : n1);
-	if (c)
-		return c;
-	return n1 - n2;
-}
-
-/*
- * Compare the values contained by the two memory cells, returning
- * negative, zero or positive if pMem1 is less than, equal to, or greater
- * than pMem2. Sorting order is NULL's first, followed by numbers (integers
- * and reals) sorted numerically, followed by text ordered by the collating
- * sequence pColl and finally blob's ordered by memcmp().
- *
- * Two NULL values are considered equal by this function.
- */
-int
-sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
-{
-	int f1, f2;
-	int combined_flags;
-
-	f1 = pMem1->flags;
-	f2 = pMem2->flags;
-	combined_flags = f1 | f2;
-
-	/* If one value is NULL, it is less than the other. If both values
-	 * are NULL, return 0.
-	 */
-	if (combined_flags & MEM_Null) {
-		return (f2 & MEM_Null) - (f1 & MEM_Null);
-	}
-
-	if ((combined_flags & MEM_Bool) != 0) {
-		if ((f1 & f2 & MEM_Bool) != 0) {
-			if (pMem1->u.b == pMem2->u.b)
-				return 0;
-			if (pMem1->u.b)
-				return 1;
-			return -1;
-		}
-		if ((f2 & MEM_Bool) != 0)
-			return +1;
-		return -1;
-	}
-
-	/* At least one of the two values is a number
-	 */
-	if ((combined_flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
-		if ((f1 & f2 & MEM_Int) != 0) {
-			if (pMem1->u.i < pMem2->u.i)
-				return -1;
-			if (pMem1->u.i > pMem2->u.i)
-				return +1;
-			return 0;
-		}
-		if ((f1 & f2 & MEM_UInt) != 0) {
-			if (pMem1->u.u < pMem2->u.u)
-				return -1;
-			if (pMem1->u.u > pMem2->u.u)
-				return +1;
-			return 0;
-		}
-		if ((f1 & f2 & MEM_Real) != 0) {
-			if (pMem1->u.r < pMem2->u.r)
-				return -1;
-			if (pMem1->u.r > pMem2->u.r)
-				return +1;
-			return 0;
-		}
-		if ((f1 & MEM_Int) != 0) {
-			if ((f2 & MEM_Real) != 0) {
-				return double_compare_nint64(pMem2->u.r,
-							     pMem1->u.i, -1);
-			} else {
-				return -1;
-			}
-		}
-		if ((f1 & MEM_UInt) != 0) {
-			if ((f2 & MEM_Real) != 0) {
-				return double_compare_uint64(pMem2->u.r,
-							     pMem1->u.u, -1);
-			} else if ((f2 & MEM_Int) != 0) {
-				return +1;
-			} else {
-				return -1;
-			}
-		}
-		if ((f1 & MEM_Real) != 0) {
-			if ((f2 & MEM_Int) != 0) {
-				return double_compare_nint64(pMem1->u.r,
-							     pMem2->u.i, 1);
-			} else if ((f2 & MEM_UInt) != 0) {
-				return double_compare_uint64(pMem1->u.r,
-							     pMem2->u.u, 1);
-			} else {
-				return -1;
-			}
-		}
-		return +1;
-	}
-
-	/* If one value is a string and the other is a blob, the string is less.
-	 * If both are strings, compare using the collating functions.
-	 */
-	if (combined_flags & MEM_Str) {
-		if ((f1 & MEM_Str) == 0) {
-			return 1;
-		}
-		if ((f2 & MEM_Str) == 0) {
-			return -1;
-		}
-		/* The collation sequence must be defined at this point, even if
-		 * the user deletes the collation sequence after the vdbe program is
-		 * compiled (this was not always the case).
-		 */
-		if (pColl) {
-			return vdbeCompareMemString(pMem1, pMem2, pColl);
-		} else {
-			size_t n = pMem1->n < pMem2->n ? pMem1->n : pMem2->n;
-			int res;
-			res = memcmp(pMem1->z, pMem2->z, n);
-			if (res == 0)
-				res = (int)pMem1->n - (int)pMem2->n;
-			return res;
-		}
-		/* If a NULL pointer was passed as the collate function, fall through
-		 * to the blob case and use memcmp().
-		 */
-	}
-
-	/* Both values must be blobs.  Compare using memcmp().  */
-	return sqlBlobCompare(pMem1, pMem2);
-}
-
 /*
  * This routine sets the value to be returned by subsequent calls to
  * sql_changes() on the database handle 'db'.
@@ -2605,291 +2325,6 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
 	return 0;
 }
 
-int
-sqlVdbeCompareMsgpack(const char **key1,
-			  struct UnpackedRecord *unpacked, int key2_idx)
-{
-	const char *aKey1 = *key1;
-	Mem *pKey2 = unpacked->aMem + key2_idx;
-	Mem mem1;
-	int rc = 0;
-	switch (mp_typeof(*aKey1)) {
-	default:{
-			/* FIXME */
-			rc = -1;
-			break;
-		}
-	case MP_NIL:{
-			rc = -((pKey2->flags & MEM_Null) == 0);
-			mp_decode_nil(&aKey1);
-			break;
-		}
-	case MP_BOOL:{
-			mem1.u.b = mp_decode_bool(&aKey1);
-			if ((pKey2->flags & MEM_Bool) != 0) {
-				if (mem1.u.b != pKey2->u.b)
-					rc = mem1.u.b ? 1 : -1;
-			} else {
-				rc = (pKey2->flags & MEM_Null) != 0 ? 1 : -1;
-			}
-			break;
-		}
-	case MP_UINT:{
-			mem1.u.u = mp_decode_uint(&aKey1);
-			if ((pKey2->flags & MEM_Int) != 0) {
-				rc = +1;
-			} else if ((pKey2->flags & MEM_UInt) != 0) {
-				if (mem1.u.u < pKey2->u.u)
-					rc = -1;
-				else if (mem1.u.u > pKey2->u.u)
-					rc = +1;
-			} else if ((pKey2->flags & MEM_Real) != 0) {
-				rc = double_compare_uint64(pKey2->u.r,
-							   mem1.u.u, -1);
-			} else if ((pKey2->flags & MEM_Null) != 0) {
-				rc = 1;
-			} else if ((pKey2->flags & MEM_Bool) != 0) {
-				rc = 1;
-			} else {
-				rc = -1;
-			}
-			break;
-		}
-	case MP_INT:{
-			mem1.u.i = mp_decode_int(&aKey1);
-			if ((pKey2->flags & MEM_UInt) != 0) {
-				rc = -1;
-			} else if ((pKey2->flags & MEM_Int) != 0) {
-				if (mem1.u.i < pKey2->u.i) {
-					rc = -1;
-				} else if (mem1.u.i > pKey2->u.i) {
-					rc = +1;
-				}
-			} else if (pKey2->flags & MEM_Real) {
-				rc = double_compare_nint64(pKey2->u.r, mem1.u.i,
-							   -1);
-			} else if ((pKey2->flags & MEM_Null) != 0) {
-				rc = 1;
-			} else if ((pKey2->flags & MEM_Bool) != 0) {
-				rc = 1;
-			} else {
-				rc = -1;
-			}
-			break;
-		}
-	case MP_FLOAT:{
-			mem1.u.r = mp_decode_float(&aKey1);
-			goto do_float;
-		}
-	case MP_DOUBLE:{
-			mem1.u.r = mp_decode_double(&aKey1);
- do_float:
-			if ((pKey2->flags & MEM_Int) != 0) {
-				rc = double_compare_nint64(mem1.u.r, pKey2->u.i,
-							   1);
-			} else if (pKey2->flags & MEM_UInt) {
-				rc = double_compare_uint64(mem1.u.r,
-							   pKey2->u.u, 1);
-			} else if (pKey2->flags & MEM_Real) {
-				if (mem1.u.r < pKey2->u.r) {
-					rc = -1;
-				} else if (mem1.u.r > pKey2->u.r) {
-					rc = +1;
-				}
-			} else if ((pKey2->flags & MEM_Null) != 0) {
-				rc = 1;
-			} else if ((pKey2->flags & MEM_Bool) != 0) {
-				rc = 1;
-			} else {
-				rc = -1;
-			}
-			break;
-		}
-	case MP_STR:{
-			if (pKey2->flags & MEM_Str) {
-				struct key_def *key_def = unpacked->key_def;
-				mem1.n = mp_decode_strl(&aKey1);
-				mem1.z = (char *)aKey1;
-				aKey1 += mem1.n;
-				struct coll *coll =
-					key_def->parts[key2_idx].coll;
-				if (coll != NULL) {
-					mem1.flags = MEM_Str;
-					rc = vdbeCompareMemString(&mem1, pKey2,
-								  coll);
-				} else {
-					goto do_bin_cmp;
-				}
-			} else {
-				rc = (pKey2->flags & MEM_Blob) ? -1 : +1;
-			}
-			break;
-		}
-	case MP_BIN:{
-			mem1.n = mp_decode_binl(&aKey1);
-			mem1.z = (char *)aKey1;
-			aKey1 += mem1.n;
- do_blob:
-			if (pKey2->flags & MEM_Blob) {
-				if (pKey2->flags & MEM_Zero) {
-					if (!isAllZero
-					    ((const char *)mem1.z, mem1.n)) {
-						rc = 1;
-					} else {
-						rc = mem1.n - pKey2->u.nZero;
-					}
-				} else {
-					int nCmp;
- do_bin_cmp:
-					nCmp = MIN(mem1.n, pKey2->n);
-					rc = memcmp(mem1.z, pKey2->z, nCmp);
-					if (rc == 0)
-						rc = mem1.n - pKey2->n;
-				}
-			} else {
-				rc = 1;
-			}
-			break;
-		}
-	case MP_ARRAY:
-	case MP_MAP:
-	case MP_EXT:{
-			mem1.z = (char *)aKey1;
-			mp_next(&aKey1);
-			mem1.n = aKey1 - (char *)mem1.z;
-			goto do_blob;
-		}
-	}
-	*key1 = aKey1;
-	return rc;
-}
-
-int
-sqlVdbeRecordCompareMsgpack(const void *key1,
-				struct UnpackedRecord *key2)
-{
-	int rc = 0;
-	u32 i, n = mp_decode_array((const char**)&key1);
-
-	n = MIN(n, key2->nField);
-
-	for (i = 0; i != n; i++) {
-		rc = sqlVdbeCompareMsgpack((const char**)&key1, key2, i);
-		if (rc != 0) {
-			if (key2->key_def->parts[i].sort_order !=
-			    SORT_ORDER_ASC) {
-				rc = -rc;
-			}
-			return rc;
-		}
-	}
-
-	key2->eqSeen = 1;
-	return key2->default_rc;
-}
-
-int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
-{
-	const char *start_buf = buf;
-	switch (mp_typeof(*buf)) {
-	case MP_ARRAY: {
-		mem->z = (char *)buf;
-		mp_next(&buf);
-		mem->n = buf - mem->z;
-		mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
-		mem->subtype = SQL_SUBTYPE_MSGPACK;
-		mem->field_type = FIELD_TYPE_ARRAY;
-		break;
-	}
-	case MP_MAP: {
-		mem->z = (char *)buf;
-		mp_next(&buf);
-		mem->n = buf - mem->z;
-		mem->flags = MEM_Blob | MEM_Ephem | MEM_Subtype;
-		mem->subtype = SQL_SUBTYPE_MSGPACK;
-		mem->field_type = FIELD_TYPE_MAP;
-		break;
-	}
-	case MP_EXT: {
-		mem->z = (char *)buf;
-		mp_next(&buf);
-		mem->n = buf - mem->z;
-		mem->flags = MEM_Blob | MEM_Ephem;
-		mem->field_type = FIELD_TYPE_VARBINARY;
-		break;
-	}
-	case MP_NIL: {
-		mp_decode_nil(&buf);
-		mem->flags = MEM_Null;
-		mem->field_type = field_type_MAX;
-		break;
-	}
-	case MP_BOOL: {
-		mem->u.b = mp_decode_bool(&buf);
-		mem->flags = MEM_Bool;
-		mem->field_type = FIELD_TYPE_BOOLEAN;
-		break;
-	}
-	case MP_UINT: {
-		uint64_t v = mp_decode_uint(&buf);
-		mem->u.u = v;
-		mem->flags = MEM_UInt;
-		mem->field_type = FIELD_TYPE_INTEGER;
-		break;
-	}
-	case MP_INT: {
-		mem->u.i = mp_decode_int(&buf);
-		mem->flags = MEM_Int;
-		mem->field_type = FIELD_TYPE_INTEGER;
-		break;
-	}
-	case MP_STR: {
-		/* XXX u32->int */
-		mem->n = (int) mp_decode_strl(&buf);
-		mem->flags = MEM_Str | MEM_Ephem;
-		mem->field_type = FIELD_TYPE_STRING;
-install_blob:
-		mem->z = (char *)buf;
-		buf += mem->n;
-		break;
-	}
-	case MP_BIN: {
-		/* XXX u32->int */
-		mem->n = (int) mp_decode_binl(&buf);
-		mem->flags = MEM_Blob | MEM_Ephem;
-		mem->field_type = FIELD_TYPE_VARBINARY;
-		goto install_blob;
-	}
-	case MP_FLOAT: {
-		mem->u.r = mp_decode_float(&buf);
-		if (sqlIsNaN(mem->u.r)) {
-			mem->flags = MEM_Null;
-			mem->field_type = field_type_MAX;
-		} else {
-			mem->flags = MEM_Real;
-			mem->field_type = FIELD_TYPE_DOUBLE;
-		}
-		break;
-	}
-	case MP_DOUBLE: {
-		mem->u.r = mp_decode_double(&buf);
-		if (sqlIsNaN(mem->u.r)) {
-			mem->flags = MEM_Null;
-			mem->field_type = field_type_MAX;
-		} else {
-			mem->flags = MEM_Real;
-			mem->field_type = FIELD_TYPE_DOUBLE;
-		}
-		break;
-	}
-	default:
-		unreachable();
-	}
-	*len = (uint32_t)(buf - start_buf);
-	return 0;
-}
-
 void
 sqlVdbeRecordUnpackMsgpack(struct key_def *key_def,	/* Information about the record format */
 			       const void *pKey,	/* The binary record */
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 1101f7205..d1a9f7e06 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -37,1057 +37,13 @@
  * name sql_value
  */
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "tarantoolInt.h"
 #include "box/schema.h"
 #include "box/tuple.h"
 #include "mpstream/mpstream.h"
 
-#ifdef SQL_DEBUG
-/*
- * Check invariants on a Mem object.
- *
- * This routine is intended for use inside of assert() statements, like
- * this:    assert( sqlVdbeCheckMemInvariants(pMem) );
- */
-int
-sqlVdbeCheckMemInvariants(Mem * p)
-{
-	/* If MEM_Dyn is set then Mem.xDel!=0.
-	 * Mem.xDel is might not be initialized if MEM_Dyn is clear.
-	 */
-	assert((p->flags & MEM_Dyn) == 0 || p->xDel != 0);
-
-	/* MEM_Dyn may only be set if Mem.szMalloc==0.  In this way we
-	 * ensure that if Mem.szMalloc>0 then it is safe to do
-	 * Mem.z = Mem.zMalloc without having to check Mem.flags&MEM_Dyn.
-	 * That saves a few cycles in inner loops.
-	 */
-	assert((p->flags & MEM_Dyn) == 0 || p->szMalloc == 0);
-
-	/* Cannot be both MEM_Int and MEM_Real at the same time */
-	assert((p->flags & (MEM_Int | MEM_Real)) != (MEM_Int | MEM_Real));
-	/* Can't be both UInt and Int at the same time.  */
-	assert((p->flags & (MEM_Int | MEM_UInt)) != (MEM_Int | MEM_UInt));
-
-	/* The szMalloc field holds the correct memory allocation size */
-	assert(p->szMalloc == 0 ||
-	       p->szMalloc == sqlDbMallocSize(p->db, p->zMalloc));
-
-	/* If p holds a string or blob, the Mem.z must point to exactly
-	 * one of the following:
-	 *
-	 *   (1) Memory in Mem.zMalloc and managed by the Mem object
-	 *   (2) Memory to be freed using Mem.xDel
-	 *   (3) An ephemeral string or blob
-	 *   (4) A static string or blob
-	 */
-	if ((p->flags & (MEM_Str | MEM_Blob)) && p->n > 0) {
-		assert(((p->szMalloc > 0 && p->z == p->zMalloc) ? 1 : 0) +
-		       ((p->flags & MEM_Dyn) != 0 ? 1 : 0) +
-		       ((p->flags & MEM_Ephem) != 0 ? 1 : 0) +
-		       ((p->flags & MEM_Static) != 0 ? 1 : 0) == 1);
-	}
-	return 1;
-}
-#endif
-
-/*
- * Make sure pMem->z points to a writable allocation of at least
- * min(n,32) bytes.
- *
- * If the bPreserve argument is true, then copy of the content of
- * pMem->z into the new allocation.  pMem must be either a string or
- * blob if bPreserve is true.  If bPreserve is false, any prior content
- * in pMem->z is discarded.
- */
-SQL_NOINLINE int
-sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
-{
-	assert(sqlVdbeCheckMemInvariants(pMem));
-	testcase(pMem->db == 0);
-
-	/* If the bPreserve flag is set to true, then the memory cell must already
-	 * contain a valid string or blob value.
-	 */
-	assert(bPreserve == 0 || pMem->flags & (MEM_Blob | MEM_Str));
-	testcase(bPreserve && pMem->z == 0);
-
-	assert(pMem->szMalloc == 0 ||
-	       pMem->szMalloc == sqlDbMallocSize(pMem->db, pMem->zMalloc));
-	if (pMem->szMalloc < n) {
-		if (n < 32)
-			n = 32;
-		if (bPreserve && pMem->szMalloc > 0 && pMem->z == pMem->zMalloc) {
-			pMem->z = pMem->zMalloc =
-			    sqlDbReallocOrFree(pMem->db, pMem->z, n);
-			bPreserve = 0;
-		} else {
-			if (pMem->szMalloc > 0)
-				sqlDbFree(pMem->db, pMem->zMalloc);
-			pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
-		}
-		if (pMem->zMalloc == 0) {
-			sqlVdbeMemSetNull(pMem);
-			pMem->z = 0;
-			pMem->szMalloc = 0;
-			return -1;
-		} else {
-			pMem->szMalloc = sqlDbMallocSize(pMem->db,
-							 pMem->zMalloc);
-		}
-	}
-
-	if (bPreserve && pMem->z && pMem->z != pMem->zMalloc) {
-		memcpy(pMem->zMalloc, pMem->z, pMem->n);
-	}
-	if ((pMem->flags & MEM_Dyn) != 0) {
-		assert(pMem->xDel != 0 && pMem->xDel != SQL_DYNAMIC);
-		pMem->xDel((void *)(pMem->z));
-	}
-
-	pMem->z = pMem->zMalloc;
-	pMem->flags &= ~(MEM_Dyn | MEM_Ephem | MEM_Static);
-	return 0;
-}
-
-/*
- * Change the pMem->zMalloc allocation to be at least szNew bytes.
- * If pMem->zMalloc already meets or exceeds the requested size, this
- * routine is a no-op.
- *
- * Any prior string or blob content in the pMem object may be discarded.
- * The pMem->xDel destructor is called, if it exists.  Though MEM_Str
- * and MEM_Blob values may be discarded, MEM_Int, MEM_Real, and MEM_Null
- * values are preserved.
- *
- * Return 0 on success or -1 if unable to complete the resizing.
- */
-int
-sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
-{
-	assert(szNew > 0);
-	assert((pMem->flags & MEM_Dyn) == 0 || pMem->szMalloc == 0);
-	if (pMem->szMalloc < szNew) {
-		return sqlVdbeMemGrow(pMem, szNew, 0);
-	}
-	assert((pMem->flags & MEM_Dyn) == 0);
-	pMem->z = pMem->zMalloc;
-	pMem->flags &= (MEM_Null | MEM_Int | MEM_Real);
-	return 0;
-}
-
-/*
- * Change pMem so that its MEM_Str or MEM_Blob value is stored in
- * MEM.zMalloc, where it can be safely written.
- *
- * Return 0 on success or -1 if malloc fails.
- */
-int
-sqlVdbeMemMakeWriteable(Mem * pMem)
-{
-	if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
-		if (ExpandBlob(pMem))
-			return -1;
-		if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
-			if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
-				return -1;
-			}
-			pMem->z[pMem->n] = 0;
-			pMem->z[pMem->n + 1] = 0;
-			pMem->flags |= MEM_Term;
-		}
-	}
-	pMem->flags &= ~MEM_Ephem;
-#ifdef SQL_DEBUG
-	pMem->pScopyFrom = 0;
-#endif
-
-	return 0;
-}
-
-/*
- * If the given Mem* has a zero-filled tail, turn it into an ordinary
- * blob stored in dynamically allocated space.
- */
-int
-sqlVdbeMemExpandBlob(Mem * pMem)
-{
-	int nByte;
-	assert(pMem->flags & MEM_Zero);
-	assert(pMem->flags & MEM_Blob);
-
-	/* Set nByte to the number of bytes required to store the expanded blob. */
-	nByte = pMem->n + pMem->u.nZero;
-	if (nByte <= 0) {
-		nByte = 1;
-	}
-	if (sqlVdbeMemGrow(pMem, nByte, 1)) {
-		return -1;
-	}
-
-	memset(&pMem->z[pMem->n], 0, pMem->u.nZero);
-	pMem->n += pMem->u.nZero;
-	pMem->flags &= ~(MEM_Zero | MEM_Term);
-	return 0;
-}
-
-/*
- * It is already known that pMem contains an unterminated string.
- * Add the zero terminator.
- */
-static SQL_NOINLINE int
-vdbeMemAddTerminator(Mem * pMem)
-{
-	if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
-		return -1;
-	}
-	pMem->z[pMem->n] = 0;
-	pMem->z[pMem->n + 1] = 0;
-	pMem->flags |= MEM_Term;
-	return 0;
-}
-
-/*
- * Make sure the given Mem is \u0000 terminated.
- */
-int
-sqlVdbeMemNulTerminate(Mem * pMem)
-{
-	testcase((pMem->flags & (MEM_Term | MEM_Str)) == (MEM_Term | MEM_Str));
-	testcase((pMem->flags & (MEM_Term | MEM_Str)) == 0);
-	if ((pMem->flags & (MEM_Term | MEM_Str)) != MEM_Str) {
-		return 0;	/* Nothing to do */
-	} else {
-		return vdbeMemAddTerminator(pMem);
-	}
-}
-
-static inline bool
-mem_has_msgpack_subtype(struct Mem *mem)
-{
-	return (mem->flags & MEM_Subtype) != 0 &&
-	       mem->subtype == SQL_SUBTYPE_MSGPACK;
-}
-
-/*
- * Add MEM_Str to the set of representations for the given Mem.  Numbers
- * are converted using sql_snprintf().  Converting a BLOB to a string
- * is a no-op.
- *
- * Existing representations MEM_Int and MEM_Real are invalidated if
- * bForce is true but are retained if bForce is false.
- *
- * A MEM_Null value will never be passed to this function. This function is
- * used for converting values to text for returning to the user (i.e. via
- * sql_value_text()), or for ensuring that values to be used as btree
- * keys are strings. In the former case a NULL pointer is returned the
- * user and the latter is an internal programming error.
- */
-int
-sqlVdbeMemStringify(Mem * pMem)
-{
-	int fg = pMem->flags;
-	int nByte = 32;
-
-	if ((fg & (MEM_Null | MEM_Str | MEM_Blob)) != 0 &&
-	    !mem_has_msgpack_subtype(pMem))
-		return 0;
-
-	assert(!(fg & MEM_Zero));
-	assert((fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool |
-		      MEM_Blob)) != 0);
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
-	/*
-	 * In case we have ARRAY/MAP we should save decoded value
-	 * before clearing pMem->z.
-	 */
-	char *value = NULL;
-	if (mem_has_msgpack_subtype(pMem)) {
-		const char *value_str = mp_str(pMem->z);
-		nByte = strlen(value_str) + 1;
-		value = region_alloc(&fiber()->gc, nByte);
-		memcpy(value, value_str, nByte);
-	}
-
-	if (sqlVdbeMemClearAndResize(pMem, nByte)) {
-		return -1;
-	}
-	if (fg & MEM_Int) {
-		sql_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
-		pMem->flags &= ~MEM_Int;
-	} else if ((fg & MEM_UInt) != 0) {
-		sql_snprintf(nByte, pMem->z, "%llu", pMem->u.u);
-		pMem->flags &= ~MEM_UInt;
-	} else if ((fg & MEM_Bool) != 0) {
-		sql_snprintf(nByte, pMem->z, "%s",
-			     SQL_TOKEN_BOOLEAN(pMem->u.b));
-		pMem->flags &= ~MEM_Bool;
-	} else if (mem_has_msgpack_subtype(pMem)) {
-		sql_snprintf(nByte, pMem->z, "%s", value);
-		pMem->flags &= ~MEM_Subtype;
-		pMem->subtype = SQL_SUBTYPE_NO;
-	} else {
-		assert(fg & MEM_Real);
-		sql_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
-		pMem->flags &= ~MEM_Real;
-	}
-	pMem->n = sqlStrlen30(pMem->z);
-	pMem->flags |= MEM_Str | MEM_Term;
-	return 0;
-}
-
-int
-sql_vdbemem_finalize(struct Mem *mem, struct func *func)
-{
-	assert(func != NULL);
-	assert(func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
-	assert(func->def->aggregate == FUNC_AGGREGATE_GROUP);
-	assert((mem->flags & MEM_Null) != 0 || func == mem->u.func);
-	sql_context ctx;
-	memset(&ctx, 0, sizeof(ctx));
-	Mem t;
-	memset(&t, 0, sizeof(t));
-	t.flags = MEM_Null;
-	t.db = mem->db;
-	t.field_type = field_type_MAX;
-	ctx.pOut = &t;
-	ctx.pMem = mem;
-	ctx.func = func;
-	((struct func_sql_builtin *)func)->finalize(&ctx);
-	assert((mem->flags & MEM_Dyn) == 0);
-	if (mem->szMalloc > 0)
-		sqlDbFree(mem->db, mem->zMalloc);
-	memcpy(mem, &t, sizeof(t));
-	return ctx.is_aborted ? -1 : 0;
-}
-
-/*
- * If the memory cell contains a value that must be freed by
- * invoking the external callback in Mem.xDel, then this routine
- * will free that value.  It also sets Mem.flags to MEM_Null.
- *
- * This is a helper routine for sqlVdbeMemSetNull() and
- * for sqlVdbeMemRelease().  Use those other routines as the
- * entry point for releasing Mem resources.
- */
-static SQL_NOINLINE void
-vdbeMemClearExternAndSetNull(Mem * p)
-{
-	assert(VdbeMemDynamic(p));
-	if (p->flags & MEM_Agg) {
-		sql_vdbemem_finalize(p, p->u.func);
-		assert((p->flags & MEM_Agg) == 0);
-		testcase(p->flags & MEM_Dyn);
-	}
-	if (p->flags & MEM_Dyn) {
-		assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
-		p->xDel((void *)p->z);
-	} else if (p->flags & MEM_Frame) {
-		VdbeFrame *pFrame = p->u.pFrame;
-		pFrame->pParent = pFrame->v->pDelFrame;
-		pFrame->v->pDelFrame = pFrame;
-	}
-	p->flags = MEM_Null;
-}
-
-/*
- * Release memory held by the Mem p, both external memory cleared
- * by p->xDel and memory in p->zMalloc.
- *
- * This is a helper routine invoked by sqlVdbeMemRelease() in
- * the unusual case where there really is memory in p that needs
- * to be freed.
- */
-static SQL_NOINLINE void
-vdbeMemClear(Mem * p)
-{
-	if (VdbeMemDynamic(p)) {
-		vdbeMemClearExternAndSetNull(p);
-	}
-	if (p->szMalloc) {
-		sqlDbFree(p->db, p->zMalloc);
-		p->szMalloc = 0;
-	}
-	p->z = 0;
-}
-
-/*
- * Release any memory resources held by the Mem.  Both the memory that is
- * free by Mem.xDel and the Mem.zMalloc allocation are freed.
- *
- * Use this routine prior to clean up prior to abandoning a Mem, or to
- * reset a Mem back to its minimum memory utilization.
- *
- * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
- * prior to inserting new content into the Mem.
- */
-void
-sqlVdbeMemRelease(Mem * p)
-{
-	assert(sqlVdbeCheckMemInvariants(p));
-	if (VdbeMemDynamic(p) || p->szMalloc) {
-		vdbeMemClear(p);
-	}
-}
-
-enum mp_type
-mem_mp_type(struct Mem *mem)
-{
-	switch (mem->flags & MEM_PURE_TYPE_MASK) {
-	case MEM_Int:
-		return MP_INT;
-	case MEM_UInt:
-		return MP_UINT;
-	case MEM_Real:
-		return MP_DOUBLE;
-	case MEM_Str:
-		return MP_STR;
-	case MEM_Blob:
-		if ((mem->flags & MEM_Subtype) == 0 ||
-		     mem->subtype != SQL_SUBTYPE_MSGPACK)
-			return MP_BIN;
-		assert(mp_typeof(*mem->z) == MP_MAP ||
-		       mp_typeof(*mem->z) == MP_ARRAY);
-		return mp_typeof(*mem->z);
-	case MEM_Bool:
-		return MP_BOOL;
-	case MEM_Null:
-		return MP_NIL;
-	default: unreachable();
-	}
-}
-
-/*
- * Convert a 64-bit IEEE double into a 64-bit signed integer.
- * If the double is out of range of a 64-bit signed integer then
- * return the closest available 64-bit signed integer.
- */
-static int
-doubleToInt64(double r, int64_t *i)
-{
-	/*
-	 * Many compilers we encounter do not define constants for the
-	 * minimum and maximum 64-bit integers, or they define them
-	 * inconsistently.  And many do not understand the "LL" notation.
-	 * So we define our own static constants here using nothing
-	 * larger than a 32-bit integer constant.
-	 */
-	static const int64_t maxInt = LARGEST_INT64;
-	static const int64_t minInt = SMALLEST_INT64;
-	if (r <= (double)minInt) {
-		*i = minInt;
-		return -1;
-	} else if (r >= (double)maxInt) {
-		*i = maxInt;
-		return -1;
-	} else {
-		*i = (int64_t) r;
-		return *i != r;
-	}
-}
-
-/*
- * Return some kind of integer value which is the best we can do
- * at representing the value that *pMem describes as an integer.
- * If pMem is an integer, then the value is exact.  If pMem is
- * a floating-point then the value returned is the integer part.
- * If pMem is a string or blob, then we make an attempt to convert
- * it into an integer and return that.  If pMem represents an
- * an SQL-NULL value, return 0.
- *
- * If pMem represents a string value, its encoding might be changed.
- */
-int
-sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
-{
-	int flags;
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-	flags = pMem->flags;
-	if (flags & MEM_Int) {
-		*i = pMem->u.i;
-		*is_neg = true;
-		return 0;
-	} else if (flags & MEM_UInt) {
-		*i = pMem->u.u;
-		*is_neg = false;
-		return 0;
-	} else if (flags & MEM_Real) {
-		*is_neg = pMem->u.r < 0;
-		return doubleToInt64(pMem->u.r, i);
-	} else if (flags & (MEM_Str)) {
-		assert(pMem->z || pMem->n == 0);
-		if (sql_atoi64(pMem->z, i, is_neg, pMem->n) == 0)
-			return 0;
-	}
-	return -1;
-}
-
-/*
- * Return the best representation of pMem that we can get into a
- * double.  If pMem is already a double or an integer, return its
- * value.  If it is a string or blob, try to convert it to a double.
- * If it is a NULL, return 0.0.
- */
-int
-sqlVdbeRealValue(Mem * pMem, double *v)
-{
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-	if (pMem->flags & MEM_Real) {
-		*v = pMem->u.r;
-		return 0;
-	} else if (pMem->flags & MEM_Int) {
-		*v = (double)pMem->u.i;
-		return 0;
-	} else if ((pMem->flags & MEM_UInt) != 0) {
-		*v = (double)pMem->u.u;
-		return 0;
-	} else if (pMem->flags & MEM_Str) {
-		if (sqlAtoF(pMem->z, v, pMem->n))
-			return 0;
-	}
-	return -1;
-}
-
-int
-mem_value_bool(const struct Mem *mem, bool *b)
-{
-	if ((mem->flags  & MEM_Bool) != 0) {
-		*b = mem->u.b;
-		return 0;
-	}
-	return -1;
-}
-
-/*
- * The MEM structure is already a MEM_Real.  Try to also make it a
- * MEM_Int if we can.
- */
-int
-mem_apply_integer_type(Mem *pMem)
-{
-	int rc;
-	i64 ix;
-	assert(pMem->flags & MEM_Real);
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
-	if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
-		mem_set_int(pMem, ix, pMem->u.r <= -1);
-	return rc;
-}
-
-/*
- * Convert pMem so that it is of type MEM_Real.
- * Invalidate any prior representations.
- */
-int
-sqlVdbeMemRealify(Mem * pMem)
-{
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-	double v;
-	if (sqlVdbeRealValue(pMem, &v))
-		return -1;
-	mem_set_double(pMem, v);
-	return 0;
-}
-
-int
-vdbe_mem_numerify(struct Mem *mem)
-{
-	if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
-		return 0;
-	if ((mem->flags & MEM_Bool) != 0) {
-		mem->u.u = mem->u.b;
-		MemSetTypeFlag(mem, MEM_UInt);
-		return 0;
-	}
-	assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
-	bool is_neg;
-	int64_t i;
-	if (sql_atoi64(mem->z, &i, &is_neg, mem->n) == 0) {
-		mem_set_int(mem, i, is_neg);
-	} else {
-		double d;
-		if (sqlAtoF(mem->z, &d, mem->n) == 0)
-			return -1;
-		mem_set_double(mem, d);
-	}
-	return 0;
-}
-
-/**
- * According to ANSI SQL string value can be converted to boolean
- * type if string consists of literal "true" or "false" and
- * number of leading and trailing spaces.
- *
- * For instance, "   tRuE  " can be successfully converted to
- * boolean value true.
- *
- * @param str String to be converted to boolean. Assumed to be
- *        null terminated.
- * @param[out] result Resulting value of cast.
- * @retval 0 If string satisfies conditions above.
- * @retval -1 Otherwise.
- */
-static int
-str_cast_to_boolean(const char *str, bool *result)
-{
-	assert(str != NULL);
-	for (; *str == ' '; str++);
-	if (strncasecmp(str, SQL_TOKEN_TRUE, strlen(SQL_TOKEN_TRUE)) == 0) {
-		*result = true;
-		str += 4;
-	} else if (strncasecmp(str, SQL_TOKEN_FALSE,
-			       strlen(SQL_TOKEN_FALSE)) == 0) {
-		*result = false;
-		str += 5;
-	} else {
-		return -1;
-	}
-	for (; *str != '\0'; ++str) {
-		if (*str != ' ')
-			return -1;
-	}
-	return 0;
-}
-
-/*
- * Cast the datatype of the value in pMem according to the type
- * @type.  Casting is different from applying type in that a cast
- * is forced.  In other words, the value is converted into the desired
- * type even if that results in loss of data.  This routine is
- * used (for example) to implement the SQL "cast()" operator.
- */
-int
-sqlVdbeMemCast(Mem * pMem, enum field_type type)
-{
-	assert(type < field_type_MAX);
-	if (pMem->flags & MEM_Null)
-		return 0;
-	switch (type) {
-	case FIELD_TYPE_SCALAR:
-		return 0;
-	case FIELD_TYPE_BOOLEAN:
-		if ((pMem->flags & MEM_Int) != 0) {
-			mem_set_bool(pMem, pMem->u.i);
-			return 0;
-		}
-		if ((pMem->flags & MEM_UInt) != 0) {
-			mem_set_bool(pMem, pMem->u.u);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Real) != 0) {
-			mem_set_bool(pMem, pMem->u.r);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Str) != 0) {
-			bool value;
-			if (str_cast_to_boolean(pMem->z, &value) != 0)
-				return -1;
-			mem_set_bool(pMem, value);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Bool) != 0)
-			return 0;
-		return -1;
-	case FIELD_TYPE_INTEGER:
-	case FIELD_TYPE_UNSIGNED:
-		if ((pMem->flags & (MEM_Blob | MEM_Str)) != 0) {
-			bool is_neg;
-			int64_t val;
-			if (sql_atoi64(pMem->z, &val, &is_neg, pMem->n) != 0)
-				return -1;
-			if (type == FIELD_TYPE_UNSIGNED && is_neg)
-				return -1;
-			mem_set_int(pMem, val, is_neg);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Bool) != 0) {
-			pMem->u.u = pMem->u.b;
-			MemSetTypeFlag(pMem, MEM_UInt);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Real) != 0) {
-			double d;
-			if (sqlVdbeRealValue(pMem, &d) != 0)
-				return -1;
-			if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
-				mem_set_int(pMem, d, d <= -1);
-				return 0;
-			}
-			if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
-				mem_set_u64(pMem, d);
-				return 0;
-			}
-			return -1;
-		}
-		if (type == FIELD_TYPE_UNSIGNED &&
-		    (pMem->flags & MEM_UInt) == 0)
-			return -1;
-		return 0;
-	case FIELD_TYPE_DOUBLE:
-		return sqlVdbeMemRealify(pMem);
-	case FIELD_TYPE_NUMBER:
-		return vdbe_mem_numerify(pMem);
-	case FIELD_TYPE_VARBINARY:
-		if ((pMem->flags & MEM_Blob) != 0)
-			return 0;
-		if ((pMem->flags & MEM_Str) != 0) {
-			MemSetTypeFlag(pMem, MEM_Str);
-			return 0;
-		}
-		return -1;
-	default:
-		assert(type == FIELD_TYPE_STRING);
-		assert(MEM_Str == (MEM_Blob >> 3));
-		if ((pMem->flags & MEM_Bool) != 0) {
-			const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
-			sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
-					 SQL_TRANSIENT);
-			return 0;
-		}
-		pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
-			sql_value_apply_type(pMem, FIELD_TYPE_STRING);
-		assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
-		pMem->flags &=
-			~(MEM_Int | MEM_UInt | MEM_Real | MEM_Blob | MEM_Zero);
-		return 0;
-	}
-}
-
-/*
- * Initialize bulk memory to be a consistent Mem object.
- *
- * The minimum amount of initialization feasible is performed.
- */
-void
-sqlVdbeMemInit(Mem * pMem, sql * db, u32 flags)
-{
-	assert((flags & ~MEM_TypeMask) == 0);
-	pMem->flags = flags;
-	pMem->db = db;
-	pMem->szMalloc = 0;
-	pMem->field_type = field_type_MAX;
-}
-
-/*
- * Delete any previous value and set the value stored in *pMem to NULL.
- *
- * This routine calls the Mem.xDel destructor to dispose of values that
- * require the destructor.  But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use sqlVdbeMemRelease(), which both calls this
- * routine to invoke the destructor and deallocates Mem.zMalloc.
- *
- * Use this routine to reset the Mem prior to insert a new value.
- *
- * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
- */
-void
-sqlVdbeMemSetNull(Mem * pMem)
-{
-	if (VdbeMemDynamic(pMem)) {
-		vdbeMemClearExternAndSetNull(pMem);
-	} else {
-		pMem->flags = MEM_Null;
-	}
-}
-
-void
-sqlValueSetNull(sql_value * p)
-{
-	sqlVdbeMemSetNull((Mem *) p);
-}
-
-void
-mem_set_ptr(struct Mem *mem, void *ptr)
-{
-	sqlVdbeMemRelease(mem);
-	mem->flags = MEM_Ptr;
-	mem->u.p = ptr;
-}
-
-/*
- * Delete any previous value and set the value to be a BLOB of length
- * n containing all zeros.
- */
-void
-sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
-{
-	sqlVdbeMemRelease(pMem);
-	pMem->flags = MEM_Blob | MEM_Zero;
-	pMem->n = 0;
-	if (n < 0)
-		n = 0;
-	pMem->u.nZero = n;
-	pMem->z = 0;
-}
-
-void
-mem_set_bool(struct Mem *mem, bool value)
-{
-	sqlVdbeMemSetNull(mem);
-	mem->u.b = value;
-	mem->flags = MEM_Bool;
-	mem->field_type = FIELD_TYPE_BOOLEAN;
-}
-
-void
-mem_set_i64(struct Mem *mem, int64_t value)
-{
-	if (VdbeMemDynamic(mem))
-		sqlVdbeMemSetNull(mem);
-	mem->u.i = value;
-	int flag = value < 0 ? MEM_Int : MEM_UInt;
-	MemSetTypeFlag(mem, flag);
-	mem->field_type = FIELD_TYPE_INTEGER;
-}
-
-void
-mem_set_u64(struct Mem *mem, uint64_t value)
-{
-	if (VdbeMemDynamic(mem))
-		sqlVdbeMemSetNull(mem);
-	mem->u.u = value;
-	MemSetTypeFlag(mem, MEM_UInt);
-	mem->field_type = FIELD_TYPE_UNSIGNED;
-}
-
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
-{
-	if (VdbeMemDynamic(mem))
-		sqlVdbeMemSetNull(mem);
-	if (is_neg) {
-		assert(value < 0);
-		mem->u.i = value;
-		MemSetTypeFlag(mem, MEM_Int);
-	} else {
-		mem->u.u = value;
-		MemSetTypeFlag(mem, MEM_UInt);
-	}
-	mem->field_type = FIELD_TYPE_INTEGER;
-}
-
-void
-mem_set_double(struct Mem *mem, double value)
-{
-	sqlVdbeMemSetNull(mem);
-	if (sqlIsNaN(value))
-		return;
-	mem->u.r = value;
-	MemSetTypeFlag(mem, MEM_Real);
-	mem->field_type = FIELD_TYPE_DOUBLE;
-}
-
-/*
- * Return true if the Mem object contains a TEXT or BLOB that is
- * too large - whose size exceeds SQL_MAX_LENGTH.
- */
-int
-sqlVdbeMemTooBig(Mem * p)
-{
-	assert(p->db != 0);
-	if (p->flags & (MEM_Str | MEM_Blob)) {
-		int n = p->n;
-		if (p->flags & MEM_Zero) {
-			n += p->u.nZero;
-		}
-		return n > p->db->aLimit[SQL_LIMIT_LENGTH];
-	}
-	return 0;
-}
-
-#ifdef SQL_DEBUG
-/*
- * This routine prepares a memory cell for modification by breaking
- * its link to a shallow copy and by marking any current shallow
- * copies of this cell as invalid.
- *
- * This is used for testing and debugging only - to make sure shallow
- * copies are not misused.
- */
-void
-sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
-{
-	int i;
-	Mem *pX;
-	for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
-		if (pX->pScopyFrom == pMem) {
-			pX->flags |= MEM_Undefined;
-			pX->pScopyFrom = 0;
-		}
-	}
-	pMem->pScopyFrom = 0;
-}
-#endif				/* SQL_DEBUG */
-
-/*
- * Make an shallow copy of pFrom into pTo.  Prior contents of
- * pTo are freed.  The pFrom->z field is not duplicated.  If
- * pFrom->z is used, then pTo->z points to the same thing as pFrom->z
- * and flags gets srcType (either MEM_Ephem or MEM_Static).
- */
-static SQL_NOINLINE void
-vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
-{
-	vdbeMemClearExternAndSetNull(pTo);
-	assert(!VdbeMemDynamic(pTo));
-	sqlVdbeMemShallowCopy(pTo, pFrom, eType);
-}
-
-void
-sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
-{
-	assert(pTo->db == pFrom->db);
-	if (VdbeMemDynamic(pTo)) {
-		vdbeClrCopy(pTo, pFrom, srcType);
-		return;
-	}
-	memcpy(pTo, pFrom, MEMCELLSIZE);
-	if ((pFrom->flags & MEM_Static) == 0) {
-		pTo->flags &= ~(MEM_Dyn | MEM_Static | MEM_Ephem);
-		assert(srcType == MEM_Ephem || srcType == MEM_Static);
-		pTo->flags |= srcType;
-	}
-}
-
-/*
- * Make a full copy of pFrom into pTo.  Prior contents of pTo are
- * freed before the copy is made.
- */
-int
-sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
-{
-	int rc = 0;
-
-	if (VdbeMemDynamic(pTo))
-		vdbeMemClearExternAndSetNull(pTo);
-	memcpy(pTo, pFrom, MEMCELLSIZE);
-	pTo->flags &= ~MEM_Dyn;
-	if (pTo->flags & (MEM_Str | MEM_Blob)) {
-		if (0 == (pFrom->flags & MEM_Static)) {
-			pTo->flags |= MEM_Ephem;
-			rc = sqlVdbeMemMakeWriteable(pTo);
-		}
-	}
-
-	return rc;
-}
-
-/*
- * Transfer the contents of pFrom to pTo. Any existing value in pTo is
- * freed. If pFrom contains ephemeral data, a copy is made.
- *
- * pFrom contains an SQL NULL when this routine returns.
- */
-void
-sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
-{
-	assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
-
-	sqlVdbeMemRelease(pTo);
-	memcpy(pTo, pFrom, sizeof(Mem));
-	pFrom->flags = MEM_Null;
-	pFrom->szMalloc = 0;
-}
-
-/*
- * Change the value of a Mem to be a string or a BLOB.
- *
- * The memory management strategy depends on the value of the xDel
- * parameter. If the value passed is SQL_TRANSIENT, then the
- * string is copied into a (possibly existing) buffer managed by the
- * Mem structure. Otherwise, any existing buffer is freed and the
- * pointer copied.
- *
- * If the string is too large (if it exceeds the SQL_LIMIT_LENGTH
- * size limit) then no memory allocation occurs.  If the string can be
- * stored without allocating memory, then it is.  If a memory allocation
- * is required to store the string, then value of pMem is unchanged.  In
- * either case, error is returned.
- */
-int
-sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
-		     const char *z,	/* String pointer */
-		     int n,	/* Bytes in string, or negative */
-		     u8 not_blob,	/* Encoding of z.  0 for BLOBs */
-		     void (*xDel) (void *)	/* Destructor function */
-    )
-{
-	int nByte = n;		/* New value for pMem->n */
-	int iLimit;		/* Maximum allowed string or blob size */
-	u16 flags = 0;		/* New value for pMem->flags */
-
-	/* If z is a NULL pointer, set pMem to contain an SQL NULL. */
-	if (!z) {
-		sqlVdbeMemSetNull(pMem);
-		return 0;
-	}
-
-	if (pMem->db) {
-		iLimit = pMem->db->aLimit[SQL_LIMIT_LENGTH];
-	} else {
-		iLimit = SQL_MAX_LENGTH;
-	}
-	flags = (not_blob == 0 ? MEM_Blob : MEM_Str);
-	if (nByte < 0) {
-		assert(not_blob != 0);
-		nByte = sqlStrlen30(z);
-		if (nByte > iLimit)
-			nByte = iLimit + 1;
-		flags |= MEM_Term;
-	}
-
-	/* The following block sets the new values of Mem.z and Mem.xDel. It
-	 * also sets a flag in local variable "flags" to indicate the memory
-	 * management (one of MEM_Dyn or MEM_Static).
-	 */
-	if (xDel == SQL_TRANSIENT) {
-		int nAlloc = nByte;
-		if (flags & MEM_Term) {
-			nAlloc += 1; //SQL_UTF8
-		}
-		if (nByte > iLimit) {
-			diag_set(ClientError, ER_SQL_EXECUTE, "string or binary"\
-				 "string is too big");
-			return -1;
-		}
-		testcase(nAlloc == 0);
-		testcase(nAlloc == 31);
-		testcase(nAlloc == 32);
-		if (sqlVdbeMemClearAndResize(pMem, MAX(nAlloc, 32))) {
-			return -1;
-		}
-		memcpy(pMem->z, z, nAlloc);
-	} else if (xDel == SQL_DYNAMIC) {
-		sqlVdbeMemRelease(pMem);
-		pMem->zMalloc = pMem->z = (char *)z;
-		pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
-	} else {
-		sqlVdbeMemRelease(pMem);
-		pMem->z = (char *)z;
-		pMem->xDel = xDel;
-		flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
-	}
-
-	pMem->n = nByte;
-	pMem->flags = flags;
-	assert((pMem->flags & (MEM_Str | MEM_Blob)) != 0);
-	if ((pMem->flags & MEM_Str) != 0)
-		pMem->field_type = FIELD_TYPE_STRING;
-	else
-		pMem->field_type = FIELD_TYPE_VARBINARY;
-
-	if (nByte > iLimit) {
-		diag_set(ClientError, ER_SQL_EXECUTE, "string or binary string"\
-			 "is too big");
-		return -1;
-	}
-
-	return 0;
-}
-
 /*
  * Move data out of a btree key or data field and into a Mem structure.
  * The data is payload from the entry that pCur is currently pointing
@@ -1153,79 +109,6 @@ sqlVdbeMemFromBtree(BtCursor * pCur,	/* Cursor pointing at record to retrieve. *
 	return rc;
 }
 
-/*
- * The pVal argument is known to be a value other than NULL.
- * Convert it into a string with encoding enc and return a pointer
- * to a zero-terminated version of that string.
- */
-static SQL_NOINLINE const void *
-valueToText(sql_value * pVal)
-{
-	assert(pVal != 0);
-	assert((pVal->flags & (MEM_Null)) == 0);
-	if ((pVal->flags & (MEM_Blob | MEM_Str)) &&
-	    !mem_has_msgpack_subtype(pVal)) {
-		if (ExpandBlob(pVal))
-			return 0;
-		pVal->flags |= MEM_Str;
-		sqlVdbeMemNulTerminate(pVal);	/* IMP: R-31275-44060 */
-	} else {
-		sqlVdbeMemStringify(pVal);
-		assert(0 == (1 & SQL_PTR_TO_INT(pVal->z)));
-	}
-	return pVal->z;
-}
-
-/* This function is only available internally, it is not part of the
- * external API. It works in a similar way to sql_value_text(),
- * except the data returned is in the encoding specified by the second
- * parameter, which must be one of SQL_UTF16BE, SQL_UTF16LE or
- * SQL_UTF8.
- *
- * (2006-02-16:)  The enc value can be or-ed with SQL_UTF16_ALIGNED.
- * If that is the case, then the result must be aligned on an even byte
- * boundary.
- */
-const void *
-sqlValueText(sql_value * pVal)
-{
-	if (!pVal)
-		return 0;
-	if ((pVal->flags & (MEM_Str | MEM_Term)) == (MEM_Str | MEM_Term)) {
-		return pVal->z;
-	}
-	if (pVal->flags & MEM_Null) {
-		return 0;
-	}
-	return valueToText(pVal);
-}
-
-const char *
-sql_value_to_diag_str(sql_value *value)
-{
-	enum mp_type mp_type = sql_value_type(value);
-	if (mp_type_is_bloblike(mp_type)) {
-		if (mem_has_msgpack_subtype(value))
-			return sqlValueText(value);
-		return "varbinary";
-	}
-	return sqlValueText(value);
-}
-
-/*
- * Create a new sql_value object.
- */
-sql_value *
-sqlValueNew(sql * db)
-{
-	Mem *p = sqlDbMallocZero(db, sizeof(*p));
-	if (p) {
-		p->flags = MEM_Null;
-		p->db = db;
-	}
-	return p;
-}
-
 /*
  * Context object passed by sqlStat4ProbeSetValue() through to
  * valueNew(). See comments above valueNew() for details.
@@ -1747,125 +630,3 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
 		sqlDbFree(aMem[0].db, pRec);
 	}
 }
-
-/*
- * Change the string value of an sql_value object
- */
-void
-sqlValueSetStr(sql_value * v,	/* Value to be set */
-		   int n,	/* Length of string z */
-		   const void *z,	/* Text of the new string */
-		   void (*xDel) (void *)	/* Destructor for the string */
-    )
-{
-	if (v)
-		sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
-}
-
-/*
- * Free an sql_value object
- */
-void
-sqlValueFree(sql_value * v)
-{
-	if (!v)
-		return;
-	sqlVdbeMemRelease((Mem *) v);
-	sqlDbFree(((Mem *) v)->db, v);
-}
-
-/*
- * The sqlValueBytes() routine returns the number of bytes in the
- * sql_value object assuming that it uses the encoding "enc".
- * The valueBytes() routine is a helper function.
- */
-static SQL_NOINLINE int
-valueBytes(sql_value * pVal)
-{
-	return valueToText(pVal) != 0 ? pVal->n : 0;
-}
-
-int
-sqlValueBytes(sql_value * pVal)
-{
-	Mem *p = (Mem *) pVal;
-	assert((p->flags & MEM_Null) == 0
-	       || (p->flags & (MEM_Str | MEM_Blob)) == 0);
-	if ((p->flags & MEM_Str) != 0) {
-		return p->n;
-	}
-	if ((p->flags & MEM_Blob) != 0) {
-		if (p->flags & MEM_Zero) {
-			return p->n + p->u.nZero;
-		} else {
-			return p->n;
-		}
-	}
-	if (p->flags & MEM_Null)
-		return 0;
-	return valueBytes(pVal);
-}
-
-void
-mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
-{
-	assert(memIsValid(var));
-	int64_t i;
-	if (var->flags & MEM_Null) {
-		mpstream_encode_nil(stream);
-	} else if (var->flags & MEM_Real) {
-		mpstream_encode_double(stream, var->u.r);
-	} else if (var->flags & MEM_Int) {
-		i = var->u.i;
-		mpstream_encode_int(stream, i);
-	} else if (var->flags & MEM_UInt) {
-		i = var->u.u;
-		mpstream_encode_uint(stream, i);
-	} else if (var->flags & MEM_Str) {
-		mpstream_encode_strn(stream, var->z, var->n);
-	} else if (var->flags & MEM_Bool) {
-		mpstream_encode_bool(stream, var->u.b);
-	} else {
-		/*
-		 * Emit BIN header iff the BLOB doesn't store
-		 * MsgPack content.
-		 */
-		if (!mem_has_msgpack_subtype(var)) {
-			uint32_t binl = var->n +
-					((var->flags & MEM_Zero) ?
-					var->u.nZero : 0);
-			mpstream_encode_binl(stream, binl);
-		}
-		mpstream_memcpy(stream, var->z, var->n);
-		if (var->flags & MEM_Zero)
-			mpstream_memset(stream, 0, var->u.nZero);
-	}
-}
-
-char *
-sql_vdbe_mem_encode_tuple(struct Mem *fields, uint32_t field_count,
-			  uint32_t *tuple_size, struct region *region)
-{
-	size_t used = region_used(region);
-	bool is_error = false;
-	struct mpstream stream;
-	mpstream_init(&stream, region, region_reserve_cb, region_alloc_cb,
-		      set_encode_error, &is_error);
-	mpstream_encode_array(&stream, field_count);
-	for (struct Mem *field = fields; field < fields + field_count; field++)
-		mpstream_encode_vdbe_mem(&stream, field);
-	mpstream_flush(&stream);
-	if (is_error) {
-		diag_set(OutOfMemory, stream.pos - stream.buf,
-			 "mpstream_flush", "stream");
-		return NULL;
-	}
-	*tuple_size = region_used(region) - used;
-	char *tuple = region_join(region, *tuple_size);
-	if (tuple == NULL) {
-		diag_set(OutOfMemory, *tuple_size, "region_join", "tuple");
-		return NULL;
-	}
-	mp_tuple_assert(tuple, tuple + *tuple_size);
-	return tuple;
-}
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index a2d681255..927f85559 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -152,6 +152,7 @@
  * the main thread to read from.
  */
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 
 /*
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index 2ee9f668c..e84bb3192 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -37,6 +37,7 @@
  * The Vdbe parse-tree explainer is also found here.
  */
 #include "sqlInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 
 /*
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 9c4f8b917..e5f35fbf8 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -40,6 +40,7 @@
 #include "coll/coll.h"
 #include "sqlInt.h"
 #include "tarantoolInt.h"
+#include "mem.h"
 #include "vdbeInt.h"
 #include "whereInt.h"
 #include "box/coll_id_cache.h"
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 3811ef3cf..0c002dbee 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -40,6 +40,7 @@
 #include "box/coll_id_cache.h"
 #include "coll/coll.h"
 #include "sqlInt.h"
+#include "mem.h"
 #include "whereInt.h"
 
 /* Forward declarations */
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 06/53] sql: remove unused MEM-related functions
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (4 preceding siblings ...)
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 05/53] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 07/53] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches
                   ` (46 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch removes remove unused MEM-related functions.

Part of #5818
---
 src/box/sql/sqlInt.h  |  4 ---
 src/box/sql/vdbe.c    |  8 ------
 src/box/sql/vdbeInt.h |  2 --
 src/box/sql/vdbeapi.c | 62 -----------------------------------------
 src/box/sql/vdbemem.c | 65 -------------------------------------------
 5 files changed, 141 deletions(-)

diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index c1a42fc2f..dd8163f5e 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -474,10 +474,6 @@ sql_column_text(sql_stmt *,
 enum mp_type
 sql_column_type(sql_stmt *stmt, int field_no);
 
-sql_value *
-sql_column_value(sql_stmt *,
-		     int iCol);
-
 /*
  * Terminate the current execution of an SQL statement and reset
  * it back to its starting state so that it can be reused.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 21abdcbd5..dbbcc1013 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -166,14 +166,6 @@ vdbeTakeBranch(int iSrcLine, u8 I, u8 M)
 }
 #endif
 
-/*
- * Convert the given register into a string if it isn't one
- * already. Return non-zero if a malloc() fails.
- */
-#define Stringify(P)						\
-	if(((P)->flags&(MEM_Str|MEM_Blob))==0 && sqlVdbeMemStringify(P)) \
-	{ goto no_mem; }
-
 /*
  * An ephemeral string value (signified by the MEM_Ephem flag) contains
  * a pointer to a dynamically allocated string where some other entity
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 089ac7575..e619f6e59 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -333,8 +333,6 @@ int sqlVdbeList(Vdbe *);
 
 int sqlVdbeHalt(Vdbe *);
 
-int sqlVdbeMemFromBtree(BtCursor *, u32, u32, Mem *);
-
 /**
  * In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
  * messagepacks are stored as binary string (i.e. featuring
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 24fa99f17..671338361 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -101,63 +101,12 @@ sql_stmt_reset(sql_stmt *pStmt)
 	return rc;
 }
 
-/*
- * Set all the parameters in the compiled SQL statement to NULL.
- */
-int
-sql_clear_bindings(sql_stmt * pStmt)
-{
-	int i;
-	int rc = 0;
-	Vdbe *p = (Vdbe *) pStmt;
-	for (i = 0; i < p->nVar; i++) {
-		sqlVdbeMemRelease(&p->aVar[i]);
-		p->aVar[i].flags = MEM_Null;
-	}
-	return rc;
-}
-
 bool
 sql_metadata_is_full()
 {
 	return current_session()->sql_flags & SQL_FullMetadata;
 }
 
-/* Make a copy of an sql_value object
- */
-sql_value *
-sql_value_dup(const sql_value * pOrig)
-{
-	sql_value *pNew;
-	if (pOrig == 0)
-		return 0;
-	pNew = sql_malloc(sizeof(*pNew));
-	if (pNew == 0)
-		return 0;
-	memset(pNew, 0, sizeof(*pNew));
-	memcpy(pNew, pOrig, MEMCELLSIZE);
-	pNew->flags &= ~MEM_Dyn;
-	pNew->db = 0;
-	if (pNew->flags & (MEM_Str | MEM_Blob)) {
-		pNew->flags &= ~(MEM_Static | MEM_Dyn);
-		pNew->flags |= MEM_Ephem;
-		if (sqlVdbeMemMakeWriteable(pNew) != 0) {
-			sqlValueFree(pNew);
-			pNew = 0;
-		}
-	}
-	return pNew;
-}
-
-/* Destroy an sql_value object previously obtained from
- * sql_value_dup().
- */
-void
-sql_value_free(sql_value * pOld)
-{
-	sqlValueFree(pOld);
-}
-
 /**************************** sql_result_  ******************************
  * The following routines are used by user-defined functions to specify
  * the function result.
@@ -566,17 +515,6 @@ sql_column_text(sql_stmt * pStmt, int i)
 	return sql_value_text(columnMem(pStmt, i));
 }
 
-sql_value *
-sql_column_value(sql_stmt * pStmt, int i)
-{
-	Mem *pOut = columnMem(pStmt, i);
-	if (pOut->flags & MEM_Static) {
-		pOut->flags &= ~MEM_Static;
-		pOut->flags |= MEM_Ephem;
-	}
-	return (sql_value *) pOut;
-}
-
 enum mp_type
 sql_column_type(sql_stmt * pStmt, int i)
 {
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index d1a9f7e06..85d620a0a 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -44,71 +44,6 @@
 #include "box/tuple.h"
 #include "mpstream/mpstream.h"
 
-/*
- * Move data out of a btree key or data field and into a Mem structure.
- * The data is payload from the entry that pCur is currently pointing
- * to.  offset and amt determine what portion of the data or key to retrieve.
- * The result is written into the pMem element.
- *
- * The pMem object must have been initialized.  This routine will use
- * pMem->zMalloc to hold the content from the btree, if possible.  New
- * pMem->zMalloc space will be allocated if necessary.  The calling routine
- * is responsible for making sure that the pMem object is eventually
- * destroyed.
- *
- * If this routine fails for any reason (malloc returns NULL or unable
- * to read from the disk) then the pMem is left in an inconsistent state.
- */
-static SQL_NOINLINE int
-vdbeMemFromBtreeResize(BtCursor * pCur,	/* Cursor pointing at record to retrieve. */
-		       u32 offset,	/* Offset from the start of data to return bytes from. */
-		       u32 amt,	/* Number of bytes to return. */
-		       Mem * pMem	/* OUT: Return data in this Mem structure. */
-    )
-{
-	int rc;
-	pMem->flags = MEM_Null;
-	if (0 == (rc = sqlVdbeMemClearAndResize(pMem, amt + 2))) {
-		sqlCursorPayload(pCur, offset, amt, pMem->z);
-		pMem->z[amt] = 0;
-		pMem->z[amt + 1] = 0;
-		pMem->flags = MEM_Blob | MEM_Term;
-		pMem->n = (int) amt;
-	}
-	return rc;
-}
-
-int
-sqlVdbeMemFromBtree(BtCursor * pCur,	/* Cursor pointing at record to retrieve. */
-			u32 offset,	/* Offset from the start of data to return bytes from. */
-			u32 amt,	/* Number of bytes to return. */
-			Mem * pMem	/* OUT: Return data in this Mem structure. */
-    )
-{
-	char *zData;		/* Data from the btree layer */
-	u32 available = 0;	/* Number of bytes available on the local btree page */
-	int rc = 0;	/* Return code */
-
-	assert(sqlCursorIsValid(pCur));
-	assert(!VdbeMemDynamic(pMem));
-	assert(pCur->curFlags & BTCF_TaCursor ||
-	       pCur->curFlags & BTCF_TEphemCursor);
-
-
-	zData = (char *)tarantoolsqlPayloadFetch(pCur, &available);
-	assert(zData != 0);
-
-	if (offset + amt <= available) {
-		pMem->z = &zData[offset];
-		pMem->flags = MEM_Blob | MEM_Ephem;
-		pMem->n = (int)amt;
-	} else {
-		rc = vdbeMemFromBtreeResize(pCur, offset, amt, pMem);
-	}
-
-	return rc;
-}
-
 /*
  * Context object passed by sqlStat4ProbeSetValue() through to
  * valueNew(). See comments above valueNew() for details.
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 07/53] sql: disable unused code in sql/vdbemem.c
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (5 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 06/53] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 08/53] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
                   ` (45 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch disables unused code in sql/vdbemem.c. It will simplify
refactoring.

Part of #5818
---
 src/box/sql/sqlInt.h  | 69 ++++++++++++++++++++++++++++---------------
 src/box/sql/vdbemem.c | 20 +++++++++++++
 2 files changed, 66 insertions(+), 23 deletions(-)

diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index dd8163f5e..b4293d961 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -4151,31 +4151,54 @@ sql_expr_new_column(struct sql *db, struct SrcList *src_list, int src_idx,
 
 int sqlExprCheckIN(Parse *, Expr *);
 
-int sqlStat4ProbeSetValue(Parse *, struct index_def *, UnpackedRecord **, Expr *, int,
-			      int, int *);
-int sqlStat4ValueFromExpr(Parse *, Expr *, enum field_type type,
-			      sql_value **);
-void sqlStat4ProbeFree(UnpackedRecord *);
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline int
+sqlStat4ProbeSetValue(struct Parse *parse, struct index_def *def,
+		      struct UnpackedRecord **rec, struct Expr *expr, int n,
+		      int i, int *out)
+{
+	(void)parse;
+	(void)def;
+	(void)rec;
+	(void)expr;
+	(void)n;
+	(void)i;
+	(void)out;
+	unreachable();
+	return 0;
+}
 
-/**
- * Extract the col_num-th column from the record.  Write
- * the column value into *res.  If *res is initially NULL
- * then a new sql_value object is allocated.
- *
- * If *res is initially NULL then the caller is responsible for
- * ensuring that the value written into *res is eventually
- * freed.
- *
- * @param db Database handle.
- * @param record Pointer to buffer containing record.
- * @param col_num Column to extract.
- * @param[out] res Extracted value.
- *
- * @retval -1 on error or 0.
- */
-int
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline int
+sqlStat4ValueFromExpr(struct Parse *parse, struct Expr *expr,
+		      enum field_type type, struct Mem **mem)
+{
+	(void)parse;
+	(void)expr;
+	(void)type;
+	(void)mem;
+	unreachable();
+	return 0;
+}
+
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline void
+sqlStat4ProbeFree(struct UnpackedRecord *rec)
+{
+	(void)rec;
+}
+
+/* TODO: Enable this function when stat-tables will be revived. */
+static inline int
 sql_stat4_column(struct sql *db, const char *record, uint32_t col_num,
-		 sql_value **res);
+		 struct Mem **res)
+{
+	(void)db;
+	(void)record;
+	(void)col_num;
+	(void)res;
+	return 0;
+}
 
 /*
  * The interface to the LEMON-generated parser
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 85d620a0a..263fe5b00 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -44,6 +44,8 @@
 #include "box/tuple.h"
 #include "mpstream/mpstream.h"
 
+#if 0
+
 /*
  * Context object passed by sqlStat4ProbeSetValue() through to
  * valueNew(). See comments above valueNew() for details.
@@ -523,6 +525,22 @@ sqlStat4ValueFromExpr(Parse * pParse,	/* Parse context */
 	return stat4ValueFromExpr(pParse, pExpr, type, 0, ppVal);
 }
 
+/**
+ * Extract the col_num-th column from the record.  Write
+ * the column value into *res.  If *res is initially NULL
+ * then a new sql_value object is allocated.
+ *
+ * If *res is initially NULL then the caller is responsible for
+ * ensuring that the value written into *res is eventually
+ * freed.
+ *
+ * @param db Database handle.
+ * @param record Pointer to buffer containing record.
+ * @param col_num Column to extract.
+ * @param[out] res Extracted value.
+ *
+ * @retval -1 on error or 0.
+ */
 int
 sql_stat4_column(struct sql *db, const char *record, uint32_t col_num,
 		 sql_value **res)
@@ -565,3 +583,5 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
 		sqlDbFree(aMem[0].db, pRec);
 	}
 }
+
+#endif
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 08/53] sql: introduce mem_str()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (6 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 07/53] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 09/53] sql: introduce mem_create() Mergen Imeev via Tarantool-patches
                   ` (44 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_str() which allows to receive value of MEM as
a string.

Part of #5818
---
 src/box/sql/func.c      |  8 ++++----
 src/box/sql/mem.c       | 45 ++++++++++++++++++++++++++++++-----------
 src/box/sql/mem.h       | 16 +++++++--------
 src/box/sql/vdbe.c      | 28 ++++++++++++-------------
 src/box/sql/vdbeaux.c   | 19 ++++-------------
 src/box/sql/vdbetrace.c | 33 ++++--------------------------
 6 files changed, 66 insertions(+), 83 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 074d41260..5ece28d6d 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -849,7 +849,7 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
 	enum mp_type mp_type = sql_value_type(argv[0]);
 	if (mp_type_is_bloblike(mp_type)) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(argv[0]), "numeric");
+			 mem_str(argv[0]), "numeric");
 		context->is_aborted = true;
 		return;
 	}
@@ -991,7 +991,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
 	UNUSED_PARAMETER(argc);
 	if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(argv[0]), "numeric");
+			 mem_str(argv[0]), "numeric");
 		context->is_aborted = true;
 		return;
 	}
@@ -1883,7 +1883,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
 	enum mp_type mp_type = sql_value_type(argv[0]);
 	if (mp_type_is_bloblike(mp_type)) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(argv[0]), "text");
+			 mem_str(argv[0]), "text");
 		context->is_aborted = true;
 		return;
 	}
@@ -1956,7 +1956,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
 	if (type != MP_DOUBLE && type != MP_INT && type != MP_UINT) {
 		if (mem_apply_numeric_type(argv[0]) != 0) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 sql_value_to_diag_str(argv[0]), "number");
+				 mem_str(argv[0]), "number");
 			context->is_aborted = true;
 			return;
 		}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 62338e1db..0ed5e38d4 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -37,6 +37,39 @@
 #include "box/tuple.h"
 #include "mpstream/mpstream.h"
 
+enum {
+	BUF_SIZE = 32,
+};
+
+const char *
+mem_str(const struct Mem *mem)
+{
+	char buf[BUF_SIZE];
+	switch (mem->flags & MEM_PURE_TYPE_MASK) {
+	case MEM_Null:
+		return "NULL";
+	case MEM_Str:
+		return tt_sprintf("%.*s", mem->n, mem->z);
+	case MEM_Int:
+		return tt_sprintf("%lld", mem->u.i);
+	case MEM_UInt:
+		return tt_sprintf("%llu", mem->u.u);
+	case MEM_Real:
+		sql_snprintf(BUF_SIZE, &buf[0], "%!.15g", mem->u.r);
+		return tt_sprintf("%s", buf);
+	case MEM_Blob:
+		if ((mem->flags & MEM_Subtype) == 0)
+			return "varbinary";
+		assert(mem->subtype == SQL_SUBTYPE_MSGPACK);
+		return mp_str(mem->z);
+	case MEM_Bool:
+		return mem->u.b ? "TRUE" : "FALSE";
+	default:
+		break;
+	}
+	return "unknown";
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -1632,18 +1665,6 @@ sqlValueText(sql_value * pVal)
 	return valueToText(pVal);
 }
 
-const char *
-sql_value_to_diag_str(sql_value *value)
-{
-	enum mp_type mp_type = sql_value_type(value);
-	if (mp_type_is_bloblike(mp_type)) {
-		if (mem_has_msgpack_subtype(value))
-			return sqlValueText(value);
-		return "varbinary";
-	}
-	return sqlValueText(value);
-}
-
 enum sql_subtype
 sql_value_subtype(sql_value * pVal)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index e78ebbe47..740c7dfba 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,6 +87,13 @@ struct Mem {
  */
 #define MEMCELLSIZE offsetof(Mem,zMalloc)
 
+/**
+ * Return a string that represent content of MEM. String is either allocated
+ * using static_alloc() of just a static variable.
+ */
+const char *
+mem_str(const struct Mem *mem);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -375,15 +382,6 @@ sql_value_text(struct Mem *);
 
 const void *sqlValueText(struct Mem *);
 
-/**
- * Return pointer to a string with the data type in the case of
- * binary data stored in @a value. Otherwise, return the result
- * of sql_value_text(). It is used due to the fact that not all
- * binary strings can be displayed correctly (e.g. contain
- * unprintable symbols).
- */
-const char *
-sql_value_to_diag_str(struct Mem *value);
 #define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
 
 enum sql_subtype
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index dbbcc1013..9b392f9cb 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1236,12 +1236,12 @@ case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
 	} else {
 		if (sqlVdbeRealValue(pIn1, &rA) != 0) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 sql_value_to_diag_str(pIn1), "numeric");
+				 mem_str(pIn1), "numeric");
 			goto abort_due_to_error;
 		}
 		if (sqlVdbeRealValue(pIn2, &rB) != 0) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 sql_value_to_diag_str(pIn2), "numeric");
+				 mem_str(pIn2), "numeric");
 			goto abort_due_to_error;
 		}
 		assert(((type1 | type2) & MEM_Real) != 0);
@@ -1517,12 +1517,12 @@ case OP_ShiftRight: {           /* same as TK_RSHIFT, in1, in2, out3 */
 	bool unused;
 	if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(pIn2), "integer");
+			 mem_str(pIn2), "integer");
 		goto abort_due_to_error;
 	}
 	if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(pIn1), "integer");
+			 mem_str(pIn1), "integer");
 		goto abort_due_to_error;
 	}
 	op = pOp->opcode;
@@ -1587,7 +1587,7 @@ case OP_MustBeInt: {            /* jump, in1 */
 		if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
 			if (pOp->p2==0) {
 				diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-					 sql_value_to_diag_str(pIn1), "integer");
+					 mem_str(pIn1), "integer");
 				goto abort_due_to_error;
 			} else {
 				goto jump_to_p2;
@@ -1644,7 +1644,7 @@ case OP_Cast: {                  /* in1 */
 	UPDATE_MAX_BLOBSIZE(pIn1);
 	if (rc == 0)
 		break;
-	diag_set(ClientError, ER_SQL_TYPE_MISMATCH, sql_value_to_diag_str(pIn1),
+	diag_set(ClientError, ER_SQL_TYPE_MISMATCH, mem_str(pIn1),
 		 field_type_strs[pOp->p2]);
 	goto abort_due_to_error;
 }
@@ -1817,7 +1817,7 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 					if (mem_apply_numeric_type(pIn3) != 0) {
 						diag_set(ClientError,
 							 ER_SQL_TYPE_MISMATCH,
-							 sql_value_to_diag_str(pIn3),
+							 mem_str(pIn3),
 							 "numeric");
 						goto abort_due_to_error;
 					}
@@ -2083,7 +2083,7 @@ case OP_Or: {             /* same as TK_OR, in1, in2, out3 */
 		v1 = pIn1->u.b;
 	} else {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(pIn1), "boolean");
+			 mem_str(pIn1), "boolean");
 		goto abort_due_to_error;
 	}
 	pIn2 = &aMem[pOp->p2];
@@ -2093,7 +2093,7 @@ case OP_Or: {             /* same as TK_OR, in1, in2, out3 */
 		v2 = pIn2->u.b;
 	} else {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(pIn2), "boolean");
+			 mem_str(pIn2), "boolean");
 		goto abort_due_to_error;
 	}
 	if (pOp->opcode==OP_And) {
@@ -2123,7 +2123,7 @@ case OP_Not: {                /* same as TK_NOT, in1, out2 */
 	if ((pIn1->flags & MEM_Null)==0) {
 		if ((pIn1->flags & MEM_Bool) == 0) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 sql_value_to_diag_str(pIn1), "boolean");
+				 mem_str(pIn1), "boolean");
 			goto abort_due_to_error;
 		}
 		mem_set_bool(pOut, ! pIn1->u.b);
@@ -2148,7 +2148,7 @@ case OP_BitNot: {             /* same as TK_BITNOT, in1, out2 */
 		bool is_neg;
 		if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 sql_value_to_diag_str(pIn1), "integer");
+				 mem_str(pIn1), "integer");
 			goto abort_due_to_error;
 		}
 		mem_set_i64(pOut, ~i);
@@ -2194,7 +2194,7 @@ case OP_IfNot: {            /* jump, in1 */
 		c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
 	} else {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(pIn1), "boolean");
+			 mem_str(pIn1), "boolean");
 		goto abort_due_to_error;
 	}
 	VdbeBranchTaken(c!=0, 2);
@@ -2374,7 +2374,7 @@ case OP_ApplyType: {
 		continue;
 type_mismatch:
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 sql_value_to_diag_str(pIn1), field_type_strs[type]);
+			 mem_str(pIn1), field_type_strs[type]);
 		goto abort_due_to_error;
 	}
 	break;
@@ -3003,7 +3003,7 @@ case OP_SeekGT: {       /* jump, in3 */
 			is_neg = i < 0;
 		} else {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 sql_value_to_diag_str(pIn3), "integer");
+				 mem_str(pIn3), "integer");
 			goto abort_due_to_error;
 		}
 		iKey = i;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 9153df1df..f270b0ed1 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1108,21 +1108,10 @@ displayP4(Op * pOp, char *zTemp, int nTemp)
 			break;
 		}
 	case P4_MEM:{
-			Mem *pMem = pOp->p4.pMem;
-			if (pMem->flags & MEM_Str) {
-				zP4 = pMem->z;
-			} else if (pMem->flags & MEM_Int) {
-				sqlXPrintf(&x, "%lld", pMem->u.i);
-			} else if (pMem->flags & MEM_UInt) {
-				sqlXPrintf(&x, "%llu", pMem->u.u);
-			} else if (pMem->flags & MEM_Real) {
-				sqlXPrintf(&x, "%.16g", pMem->u.r);
-			} else if (pMem->flags & MEM_Null) {
-				zP4 = "NULL";
-			} else {
-				assert(pMem->flags & MEM_Blob);
-				zP4 = "(binary string)";
-			}
+			const char *value = mem_str(pOp->p4.pMem);
+			uint32_t size = MIN((int)strlen(value), nTemp - 1);
+			memcpy(zP4, value, size);
+			zP4[size] = '\0';
 			break;
 		}
 	case P4_INTARRAY:{
diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
index e84bb3192..eceaa953b 100644
--- a/src/box/sql/vdbetrace.c
+++ b/src/box/sql/vdbetrace.c
@@ -97,8 +97,6 @@ sqlVdbeExpandSql(Vdbe * p,	/* The prepared statement being evaluated */
 	int nextIndex = 1;	/* Index of next ? host parameter */
 	int n;			/* Length of a token prefix */
 	int nToken;		/* Length of the parameter token */
-	int i;			/* Loop counter */
-	Mem *pVar;		/* Value of a host parameter */
 	StrAccum out;		/* Accumulate the output here */
 	char zBase[100];	/* Initial working space */
 
@@ -147,33 +145,10 @@ sqlVdbeExpandSql(Vdbe * p,	/* The prepared statement being evaluated */
 			zRawSql += nToken;
 			nextIndex = idx + 1;
 			assert(idx > 0 && idx <= p->nVar);
-			pVar = &p->aVar[idx - 1];
-			if (pVar->flags & MEM_Null) {
-				sqlStrAccumAppend(&out, "NULL", 4);
-			} else if (pVar->flags & MEM_Int) {
-				sqlXPrintf(&out, "%lld", pVar->u.i);
-			} else if (pVar->flags & MEM_UInt) {
-				sqlXPrintf(&out, "%llu", pVar->u.u);
-			} else if (pVar->flags & MEM_Real) {
-				sqlXPrintf(&out, "%!.15g", pVar->u.r);
-			} else if (pVar->flags & MEM_Str) {
-				int nOut;	/* Number of bytes of the string text to include in output */
-				nOut = pVar->n;
-				sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
-			} else if (pVar->flags & MEM_Zero) {
-				sqlXPrintf(&out, "zeroblob(%d)",
-					       pVar->u.nZero);
-			} else {
-				int nOut;	/* Number of bytes of the blob to include in output */
-				assert(pVar->flags & MEM_Blob);
-				sqlStrAccumAppend(&out, "x'", 2);
-				nOut = pVar->n;
-				for (i = 0; i < nOut; i++) {
-					sqlXPrintf(&out, "%02x",
-						       pVar->z[i] & 0xff);
-				}
-				sqlStrAccumAppend(&out, "'", 1);
-			}
+			const char *value = mem_str(&p->aVar[idx - 1]);
+			uint32_t size = MIN(strlen(value), sizeof(zBase) - 1);
+			memcpy(zBase, value, size);
+			zBase[size] = '\0';
 		}
 	}
 	if (out.accError)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 09/53] sql: introduce mem_create()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (7 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 08/53] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 10/53] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches
                   ` (43 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_create(). This function should be used to
initialize a MEM. MEM should be initialized before usage.

Part of #5818
---
 src/box/sql/func.c    |  2 +-
 src/box/sql/mem.c     | 49 +++++++++++++++++--------------------------
 src/box/sql/mem.h     | 14 ++++++-------
 src/box/sql/vdbe.c    |  2 +-
 src/box/sql/vdbeaux.c |  8 +++++--
 5 files changed, 33 insertions(+), 42 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 5ece28d6d..5e4eff931 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -95,7 +95,7 @@ vdbemem_alloc_on_region(uint32_t count)
 	}
 	memset(ret, 0, count * sizeof(*ret));
 	for (uint32_t i = 0; i < count; i++) {
-		sqlVdbeMemInit(&ret[i], sql_get(), MEM_Null);
+		mem_create(&ret[i]);
 		assert(memIsValid(&ret[i]));
 	}
 	return ret;
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 0ed5e38d4..242360a66 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -70,6 +70,25 @@ mem_str(const struct Mem *mem)
 	return "unknown";
 }
 
+void
+mem_create(struct Mem *mem)
+{
+	mem->flags = MEM_Null;
+	mem->subtype = SQL_SUBTYPE_NO;
+	mem->field_type = field_type_MAX;
+	mem->n = 0;
+	mem->z = NULL;
+	mem->zMalloc = NULL;
+	mem->szMalloc = 0;
+	mem->uTemp = 0;
+	mem->db = sql_get();
+	mem->xDel = NULL;
+#ifdef SQL_DEBUG
+	mem->pScopyFrom = NULL;
+	mem->pFiller = NULL;
+#endif
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -1339,21 +1358,6 @@ sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
 	return 0;
 }
 
-/*
- * Initialize bulk memory to be a consistent Mem object.
- *
- * The minimum amount of initialization feasible is performed.
- */
-void
-sqlVdbeMemInit(Mem * pMem, sql * db, u32 flags)
-{
-	assert((flags & ~MEM_TypeMask) == 0);
-	pMem->flags = flags;
-	pMem->db = db;
-	pMem->szMalloc = 0;
-	pMem->field_type = field_type_MAX;
-}
-
 /*
  * Delete any previous value and set the value stored in *pMem to NULL.
  *
@@ -1438,21 +1442,6 @@ sqlValueNew(sql * db)
 	return p;
 }
 
-void
-initMemArray(Mem * p, int N, sql * db, u32 flags)
-{
-	while ((N--) > 0) {
-		p->db = db;
-		p->flags = flags;
-		p->szMalloc = 0;
-		p->field_type = field_type_MAX;
-#ifdef SQL_DEBUG
-		p->pScopyFrom = 0;
-#endif
-		p++;
-	}
-}
-
 void
 releaseMemArray(Mem * p, int N)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 740c7dfba..9a3b275fc 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -94,6 +94,12 @@ struct Mem {
 const char *
 mem_str(const struct Mem *mem);
 
+/**
+ * Initialize MEM and set NULL.
+ */
+void
+mem_create(struct Mem *mem);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -321,8 +327,6 @@ mem_set_double(struct Mem *mem, double value);
 int
 sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
 void
-sqlVdbeMemInit(struct Mem *, sql *, u32);
-void
 sqlVdbeMemSetNull(struct Mem *);
 void
 sqlVdbeMemSetZeroBlob(struct Mem *, int);
@@ -332,12 +336,6 @@ void sqlValueSetNull(struct Mem *);
 void sqlValueFree(struct Mem *);
 struct Mem *sqlValueNew(struct sql *);
 
-/*
- * Initialize an array of N Mem element.
- */
-void
-initMemArray(Mem * p, int N, sql * db, u32 flags);
-
 /*
  * Release an array of N Mem elements
  */
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 9b392f9cb..5378bc92b 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4710,7 +4710,7 @@ case OP_AggStep: {
 #endif
 
 	pMem->n++;
-	sqlVdbeMemInit(&t, db, MEM_Null);
+	mem_create(&t);
 	pCtx->pOut = &t;
 	pCtx->is_aborted = false;
 	pCtx->skipFlag = 0;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index f270b0ed1..55d4fc14f 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1565,9 +1565,13 @@ sqlVdbeMakeReady(Vdbe * p,	/* The VDBE */
 	} else {
 		p->nCursor = nCursor;
 		p->nVar = (ynVar) nVar;
-		initMemArray(p->aVar, nVar, db, MEM_Null);
+		for (int i = 0; i < nVar; ++i)
+			mem_create(&p->aVar[i]);
 		p->nMem = nMem;
-		initMemArray(p->aMem, nMem, db, MEM_Undefined);
+		for (int i = 0; i < nMem; ++i) {
+			mem_create(&p->aMem[i]);
+			p->aMem[i].flags = MEM_Undefined;
+		}
 		memset(p->apCsr, 0, nCursor * sizeof(VdbeCursor *));
 	}
 	sqlVdbeRewind(p);
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 10/53] sql: introduce mem_destroy()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (8 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 09/53] sql: introduce mem_create() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 11/53] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
                   ` (42 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_destroy(). This function should be used to
free and destroing all objects owned by MEM, if necessary.

Part of #5818
---
 src/box/sql/func.c    |   6 +-
 src/box/sql/mem.c     | 171 +++++++++++++-----------------------------
 src/box/sql/mem.h     |   7 +-
 src/box/sql/vdbe.c    |   6 +-
 src/box/sql/vdbeInt.h |   2 +-
 src/box/sql/vdbeapi.c |   4 +-
 src/box/sql/vdbemem.c |   2 +-
 7 files changed, 66 insertions(+), 132 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 5e4eff931..e600a9800 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -301,7 +301,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
 	return (struct sql_value *)val;
 error:
 	for (int i = 0; i < argc; i++)
-		sqlVdbeMemRelease(&val[i]);
+		mem_destroy(&val[i]);
 	region_truncate(region, region_svp);
 	return NULL;
 }
@@ -373,7 +373,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
 	return (struct sql_value *) val;
 error:
 	for (int i = 0; i < port->size; i++)
-		sqlVdbeMemRelease(&val[i]);
+		mem_destroy(&val[i]);
 	region_truncate(region, region_svp);
 	return NULL;
 }
@@ -2102,7 +2102,7 @@ minMaxFinalize(sql_context * context)
 		if (pRes->flags) {
 			sql_result_value(context, pRes);
 		}
-		sqlVdbeMemRelease(pRes);
+		mem_destroy(pRes);
 	}
 }
 
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 242360a66..ec6aaab64 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -89,6 +89,38 @@ mem_create(struct Mem *mem)
 #endif
 }
 
+static inline void
+mem_clear(struct Mem *mem)
+{
+	if ((mem->flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) != 0) {
+		if ((mem->flags & MEM_Agg) != 0)
+			sql_vdbemem_finalize(mem, mem->u.func);
+		assert((mem->flags & MEM_Agg) == 0);
+		if ((mem->flags & MEM_Dyn) != 0) {
+			assert(mem->xDel != SQL_DYNAMIC && mem->xDel != NULL);
+			mem->xDel((void *)mem->z);
+		} else if ((mem->flags & MEM_Frame) != 0) {
+			struct VdbeFrame *frame = mem->u.pFrame;
+			frame->pParent = frame->v->pDelFrame;
+			frame->v->pDelFrame = frame;
+		}
+	}
+	mem->flags = MEM_Null;
+	mem->field_type = field_type_MAX;
+}
+
+void
+mem_destroy(struct Mem *mem)
+{
+	mem_clear(mem);
+	if (mem->szMalloc > 0)
+		sqlDbFree(mem->db, mem->zMalloc);
+	mem->n = 0;
+	mem->z = NULL;
+	mem->szMalloc = 0;
+	mem->zMalloc = NULL;
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -200,56 +232,6 @@ vdbeMemAddTerminator(Mem * pMem)
 	return 0;
 }
 
-/*
- * If the memory cell contains a value that must be freed by
- * invoking the external callback in Mem.xDel, then this routine
- * will free that value.  It also sets Mem.flags to MEM_Null.
- *
- * This is a helper routine for sqlVdbeMemSetNull() and
- * for sqlVdbeMemRelease().  Use those other routines as the
- * entry point for releasing Mem resources.
- */
-static SQL_NOINLINE void
-vdbeMemClearExternAndSetNull(Mem * p)
-{
-	assert(VdbeMemDynamic(p));
-	if (p->flags & MEM_Agg) {
-		sql_vdbemem_finalize(p, p->u.func);
-		assert((p->flags & MEM_Agg) == 0);
-		testcase(p->flags & MEM_Dyn);
-	}
-	if (p->flags & MEM_Dyn) {
-		assert(p->xDel != SQL_DYNAMIC && p->xDel != 0);
-		p->xDel((void *)p->z);
-	} else if (p->flags & MEM_Frame) {
-		VdbeFrame *pFrame = p->u.pFrame;
-		pFrame->pParent = pFrame->v->pDelFrame;
-		pFrame->v->pDelFrame = pFrame;
-	}
-	p->flags = MEM_Null;
-}
-
-/*
- * Release memory held by the Mem p, both external memory cleared
- * by p->xDel and memory in p->zMalloc.
- *
- * This is a helper routine invoked by sqlVdbeMemRelease() in
- * the unusual case where there really is memory in p that needs
- * to be freed.
- */
-static SQL_NOINLINE void
-vdbeMemClear(Mem * p)
-{
-	if (VdbeMemDynamic(p)) {
-		vdbeMemClearExternAndSetNull(p);
-	}
-	if (p->szMalloc) {
-		sqlDbFree(p->db, p->zMalloc);
-		p->szMalloc = 0;
-	}
-	p->z = 0;
-}
-
 /*
  * Make an shallow copy of pFrom into pTo.  Prior contents of
  * pTo are freed.  The pFrom->z field is not duplicated.  If
@@ -259,7 +241,7 @@ vdbeMemClear(Mem * p)
 static SQL_NOINLINE void
 vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
 {
-	vdbeMemClearExternAndSetNull(pTo);
+	mem_clear(pTo);
 	assert(!VdbeMemDynamic(pTo));
 	sqlVdbeMemShallowCopy(pTo, pFrom, eType);
 }
@@ -1148,7 +1130,7 @@ sqlVdbeMemGrow(Mem * pMem, int n, int bPreserve)
 			pMem->zMalloc = sqlDbMallocRaw(pMem->db, n);
 		}
 		if (pMem->zMalloc == 0) {
-			sqlVdbeMemSetNull(pMem);
+			mem_clear(pMem);
 			pMem->z = 0;
 			pMem->szMalloc = 0;
 			return -1;
@@ -1200,7 +1182,7 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
 void
 mem_set_bool(struct Mem *mem, bool value)
 {
-	sqlVdbeMemSetNull(mem);
+	mem_clear(mem);
 	mem->u.b = value;
 	mem->flags = MEM_Bool;
 	mem->field_type = FIELD_TYPE_BOOLEAN;
@@ -1209,7 +1191,7 @@ mem_set_bool(struct Mem *mem, bool value)
 void
 mem_set_ptr(struct Mem *mem, void *ptr)
 {
-	sqlVdbeMemRelease(mem);
+	mem_destroy(mem);
 	mem->flags = MEM_Ptr;
 	mem->u.p = ptr;
 }
@@ -1217,8 +1199,7 @@ mem_set_ptr(struct Mem *mem, void *ptr)
 void
 mem_set_i64(struct Mem *mem, int64_t value)
 {
-	if (VdbeMemDynamic(mem))
-		sqlVdbeMemSetNull(mem);
+	mem_clear(mem);
 	mem->u.i = value;
 	int flag = value < 0 ? MEM_Int : MEM_UInt;
 	MemSetTypeFlag(mem, flag);
@@ -1228,8 +1209,7 @@ mem_set_i64(struct Mem *mem, int64_t value)
 void
 mem_set_u64(struct Mem *mem, uint64_t value)
 {
-	if (VdbeMemDynamic(mem))
-		sqlVdbeMemSetNull(mem);
+	mem_clear(mem);
 	mem->u.u = value;
 	MemSetTypeFlag(mem, MEM_UInt);
 	mem->field_type = FIELD_TYPE_UNSIGNED;
@@ -1238,8 +1218,7 @@ mem_set_u64(struct Mem *mem, uint64_t value)
 void
 mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
 {
-	if (VdbeMemDynamic(mem))
-		sqlVdbeMemSetNull(mem);
+	mem_clear(mem);
 	if (is_neg) {
 		assert(value < 0);
 		mem->u.i = value;
@@ -1254,7 +1233,7 @@ mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
 void
 mem_set_double(struct Mem *mem, double value)
 {
-	sqlVdbeMemSetNull(mem);
+	mem_clear(mem);
 	if (sqlIsNaN(value))
 		return;
 	mem->u.r = value;
@@ -1291,7 +1270,7 @@ sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
 
 	/* If z is a NULL pointer, set pMem to contain an SQL NULL. */
 	if (!z) {
-		sqlVdbeMemSetNull(pMem);
+		mem_clear(pMem);
 		return 0;
 	}
 
@@ -1331,11 +1310,11 @@ sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
 		}
 		memcpy(pMem->z, z, nAlloc);
 	} else if (xDel == SQL_DYNAMIC) {
-		sqlVdbeMemRelease(pMem);
+		mem_destroy(pMem);
 		pMem->zMalloc = pMem->z = (char *)z;
 		pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
 	} else {
-		sqlVdbeMemRelease(pMem);
+		mem_destroy(pMem);
 		pMem->z = (char *)z;
 		pMem->xDel = xDel;
 		flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
@@ -1363,21 +1342,17 @@ sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
  *
  * This routine calls the Mem.xDel destructor to dispose of values that
  * require the destructor.  But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use sqlVdbeMemRelease(), which both calls this
+ * To free all resources, use mem_destroy(), which both calls this
  * routine to invoke the destructor and deallocates Mem.zMalloc.
  *
  * Use this routine to reset the Mem prior to insert a new value.
  *
- * Use sqlVdbeMemRelease() to complete erase the Mem prior to abandoning it.
+ * Use mem_destroy() to complete erase the Mem prior to abandoning it.
  */
 void
 sqlVdbeMemSetNull(Mem * pMem)
 {
-	if (VdbeMemDynamic(pMem)) {
-		vdbeMemClearExternAndSetNull(pMem);
-	} else {
-		pMem->flags = MEM_Null;
-	}
+	mem_clear(pMem);
 }
 
 /*
@@ -1387,7 +1362,7 @@ sqlVdbeMemSetNull(Mem * pMem)
 void
 sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
 {
-	sqlVdbeMemRelease(pMem);
+	mem_destroy(pMem);
 	pMem->flags = MEM_Blob | MEM_Zero;
 	pMem->n = 0;
 	if (n < 0)
@@ -1424,7 +1399,7 @@ sqlValueFree(sql_value * v)
 {
 	if (!v)
 		return;
-	sqlVdbeMemRelease((Mem *) v);
+	mem_destroy((Mem *) v);
 	sqlDbFree(((Mem *) v)->db, v);
 }
 
@@ -1447,34 +1422,10 @@ releaseMemArray(Mem * p, int N)
 {
 	if (p && N) {
 		Mem *pEnd = &p[N];
-		sql *db = p->db;
 		do {
 			assert((&p[1]) == pEnd || p[0].db == p[1].db);
 			assert(sqlVdbeCheckMemInvariants(p));
-
-			/* This block is really an inlined version of sqlVdbeMemRelease()
-			 * that takes advantage of the fact that the memory cell value is
-			 * being set to NULL after releasing any dynamic resources.
-			 *
-			 * The justification for duplicating code is that according to
-			 * callgrind, this causes a certain test case to hit the CPU 4.7
-			 * percent less (x86 linux, gcc version 4.1.2, -O6) than if
-			 * sqlMemRelease() were called from here. With -O2, this jumps
-			 * to 6.6 percent. The test case is inserting 1000 rows into a table
-			 * with no indexes using a single prepared INSERT statement, bind()
-			 * and reset(). Inserts are grouped into a transaction.
-			 */
-			testcase(p->flags & MEM_Agg);
-			testcase(p->flags & MEM_Dyn);
-			testcase(p->flags & MEM_Frame);
-			if (p->
-			    flags & (MEM_Agg | MEM_Dyn | MEM_Frame)) {
-				sqlVdbeMemRelease(p);
-			} else if (p->szMalloc) {
-				sqlDbFree(db, p->zMalloc);
-				p->szMalloc = 0;
-			}
-
+			mem_destroy(p);
 			p->flags = MEM_Undefined;
 		} while ((++p) < pEnd);
 	}
@@ -1916,8 +1867,7 @@ sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
 {
 	int rc = 0;
 
-	if (VdbeMemDynamic(pTo))
-		vdbeMemClearExternAndSetNull(pTo);
+	mem_clear(pTo);
 	memcpy(pTo, pFrom, MEMCELLSIZE);
 	pTo->flags &= ~MEM_Dyn;
 	if (pTo->flags & (MEM_Str | MEM_Blob)) {
@@ -1957,7 +1907,7 @@ sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
 {
 	assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
 
-	sqlVdbeMemRelease(pTo);
+	mem_destroy(pTo);
 	memcpy(pTo, pFrom, sizeof(Mem));
 	pFrom->flags = MEM_Null;
 	pFrom->szMalloc = 0;
@@ -1992,25 +1942,6 @@ sqlVdbeMemMakeWriteable(Mem * pMem)
 	return 0;
 }
 
-/*
- * Release any memory resources held by the Mem.  Both the memory that is
- * free by Mem.xDel and the Mem.zMalloc allocation are freed.
- *
- * Use this routine prior to clean up prior to abandoning a Mem, or to
- * reset a Mem back to its minimum memory utilization.
- *
- * Use sqlVdbeMemSetNull() to release just the Mem.xDel space
- * prior to inserting new content into the Mem.
- */
-void
-sqlVdbeMemRelease(Mem * p)
-{
-	assert(sqlVdbeCheckMemInvariants(p));
-	if (VdbeMemDynamic(p) || p->szMalloc) {
-		vdbeMemClear(p);
-	}
-}
-
 int
 sql_vdbemem_finalize(struct Mem *mem, struct func *func)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9a3b275fc..5ac02c002 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -100,6 +100,12 @@ mem_str(const struct Mem *mem);
 void
 mem_create(struct Mem *mem);
 
+/**
+ * Destroy MEM and set NULL.
+ */
+void
+mem_destroy(struct Mem *mem);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -426,7 +432,6 @@ int sqlVdbeMemCopy(Mem *, const Mem *);
 void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
 void sqlVdbeMemMove(Mem *, Mem *);
 int sqlVdbeMemMakeWriteable(Mem *);
-void sqlVdbeMemRelease(Mem * p);
 
 /**
  * Memory cell mem contains the context of an aggregate function.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 5378bc92b..12712efb4 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2471,7 +2471,7 @@ case OP_MakeRecord: {
 		 * to be passed to Tarantool. Before that, make
 		 * sure previously allocated memory has gone.
 		 */
-		sqlVdbeMemRelease(pOut);
+		mem_destroy(pOut);
 		pOut->flags = MEM_Blob | MEM_Ephem;
 		pOut->n = tuple_size;
 		pOut->z = tuple;
@@ -4425,7 +4425,7 @@ case OP_Program: {        /* jump */
 		if (!pFrame) {
 			goto no_mem;
 		}
-		sqlVdbeMemRelease(pRt);
+		mem_destroy(pRt);
 		pRt->flags = MEM_Frame;
 		pRt->u.pFrame = pFrame;
 
@@ -4718,7 +4718,7 @@ case OP_AggStep: {
 	struct func_sql_builtin *func = (struct func_sql_builtin *)pCtx->func;
 	func->call(pCtx, pCtx->argc, pCtx->argv);
 	if (pCtx->is_aborted) {
-		sqlVdbeMemRelease(&t);
+		mem_destroy(&t);
 		goto abort_due_to_error;
 	}
 	assert(t.flags==MEM_Null);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index e619f6e59..338b8bc96 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -130,7 +130,7 @@ struct VdbeCursor {
  * is linked into the Vdbe.pDelFrame list. The contents of the Vdbe.pDelFrame
  * list is deleted when the VM is reset in VdbeHalt(). The reason for doing
  * this instead of deleting the VdbeFrame immediately is to avoid recursive
- * calls to sqlVdbeMemRelease() when the memory cells belonging to the
+ * calls to mem_destroy() when the memory cells belonging to the
  * child frame are released.
  *
  * The currently executing frame is stored in Vdbe.pFrame. Vdbe.pFrame is
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 671338361..a195f8dfd 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -681,9 +681,7 @@ vdbeUnbind(Vdbe * p, int i)
 	}
 	i--;
 	pVar = &p->aVar[i];
-	sqlVdbeMemRelease(pVar);
-	pVar->flags = MEM_Null;
-	pVar->field_type = field_type_MAX;
+	mem_destroy(pVar);
 	return 0;
 }
 
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 263fe5b00..bb87bb902 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -579,7 +579,7 @@ sqlStat4ProbeFree(UnpackedRecord * pRec)
 		int part_count = pRec->key_def->part_count;
 		struct Mem *aMem = pRec->aMem;
 		for (int i = 0; i < part_count; i++)
-			sqlVdbeMemRelease(&aMem[i]);
+			mem_destroy(&aMem[i]);
 		sqlDbFree(aMem[0].db, pRec);
 	}
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 11/53] sql: introduce mem_is_*() functions()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (9 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 10/53] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 12/53] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
                   ` (41 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_is_*() functions that allows to check current
MEM state.

Part of #5818
---
 src/box/sql/func.c      |  40 ++++---
 src/box/sql/mem.c       | 139 +++++++++++++++++++++++-
 src/box/sql/mem.h       |  69 ++++++++++--
 src/box/sql/select.c    |   2 +-
 src/box/sql/vdbe.c      | 228 +++++++++++++++++++---------------------
 src/box/sql/vdbeInt.h   |  11 --
 src/box/sql/vdbeapi.c   |   4 +-
 src/box/sql/vdbeaux.c   |   6 +-
 src/box/sql/vdbesort.c  |   2 +-
 src/box/sql/whereexpr.c |   3 +-
 10 files changed, 332 insertions(+), 172 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index e600a9800..81b537d9b 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -398,10 +398,10 @@ minmaxFunc(sql_context * context, int argc, sql_value ** argv)
 	pColl = sqlGetFuncCollSeq(context);
 	assert(mask == -1 || mask == 0);
 	iBest = 0;
-	if (sql_value_is_null(argv[0]))
+	if (mem_is_null(argv[0]))
 		return;
 	for (i = 1; i < argc; i++) {
-		if (sql_value_is_null(argv[i]))
+		if (mem_is_null(argv[i]))
 			return;
 		if ((sqlMemCompare(argv[iBest], argv[i], pColl) ^ mask) >=
 		    0) {
@@ -736,8 +736,8 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 		context->is_aborted = true;
 		return;
 	}
-	if (sql_value_is_null(argv[1])
-	    || (argc == 3 && sql_value_is_null(argv[2]))
+	if (mem_is_null(argv[1])
+	    || (argc == 3 && mem_is_null(argv[2]))
 	    ) {
 		return;
 	}
@@ -838,16 +838,15 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
 		return;
 	}
 	if (argc == 2) {
-		if (sql_value_is_null(argv[1]))
+		if (mem_is_null(argv[1]))
 			return;
 		n = sql_value_int(argv[1]);
 		if (n < 0)
 			n = 0;
 	}
-	if (sql_value_is_null(argv[0]))
+	if (mem_is_null(argv[0]))
 		return;
-	enum mp_type mp_type = sql_value_type(argv[0]);
-	if (mp_type_is_bloblike(mp_type)) {
+	if (mem_is_binary(argv[0])) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 			 mem_str(argv[0]), "numeric");
 		context->is_aborted = true;
@@ -907,8 +906,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv)   \
 	const char *z2;                                                        \
 	int n;                                                                 \
 	UNUSED_PARAMETER(argc);                                                \
-	int arg_type = sql_value_type(argv[0]);                                \
-	if (mp_type_is_bloblike(arg_type)) {                                   \
+	if (mem_is_binary(argv[0])) {                                          \
 		diag_set(ClientError, ER_INCONSISTENT_TYPES, "text",           \
 			 "varbinary");                                         \
 		context->is_aborted = true;                                    \
@@ -989,7 +987,7 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
 	unsigned char *p;
 	assert(argc == 1);
 	UNUSED_PARAMETER(argc);
-	if (mp_type_is_bloblike(sql_value_type(argv[0]))) {
+	if (mem_is_binary(argv[0])) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 			 mem_str(argv[0]), "numeric");
 		context->is_aborted = true;
@@ -1439,7 +1437,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 		break;
 	}
 	default:{
-			assert(sql_value_is_null(argv[0]));
+			assert(mem_is_null(argv[0]));
 			sql_result_text(context, "NULL", 4, SQL_STATIC);
 			break;
 		}
@@ -1578,13 +1576,13 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
 	assert(zStr == sql_value_text(argv[0]));	/* No encoding change */
 	zPattern = sql_value_text(argv[1]);
 	if (zPattern == 0) {
-		assert(sql_value_is_null(argv[1])
+		assert(mem_is_null(argv[1])
 		       || sql_context_db_handle(context)->mallocFailed);
 		return;
 	}
 	nPattern = sql_value_bytes(argv[1]);
 	if (nPattern == 0) {
-		assert(! sql_value_is_null(argv[1]));
+		assert(! mem_is_null(argv[1]));
 		sql_result_value(context, argv[0]);
 		return;
 	}
@@ -1748,10 +1746,9 @@ trim_func_one_arg(struct sql_context *context, sql_value *arg)
 {
 	/* In case of VARBINARY type default trim octet is X'00'. */
 	const unsigned char *default_trim;
-	enum mp_type val_type = sql_value_type(arg);
-	if (val_type == MP_NIL)
+	if (mem_is_null(arg))
 		return;
-	if (mp_type_is_bloblike(val_type))
+	if (mem_is_binary(arg))
 		default_trim = (const unsigned char *) "\0";
 	else
 		default_trim = (const unsigned char *) " ";
@@ -1880,8 +1877,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
 		1, 2, 6, 2, 3, 0, 1, 0, 2, 0, 2, 0, 0, 0, 0, 0,
 	};
 	assert(argc == 1);
-	enum mp_type mp_type = sql_value_type(argv[0]);
-	if (mp_type_is_bloblike(mp_type)) {
+	if (mem_is_binary(argv[0])) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 			 mem_str(argv[0]), "text");
 		context->is_aborted = true;
@@ -2039,7 +2035,7 @@ countStep(sql_context * context, int argc, sql_value ** argv)
 		return;
 	}
 	p = sql_aggregate_context(context, sizeof(*p));
-	if ((argc == 0 || ! sql_value_is_null(argv[0])) && p) {
+	if ((argc == 0 || ! mem_is_null(argv[0])) && p) {
 		p->n++;
 	}
 }
@@ -2068,7 +2064,7 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
 	if (!pBest)
 		return;
 
-	if (sql_value_is_null(argv[0])) {
+	if (mem_is_null(argv[0])) {
 		if (pBest->flags)
 			sqlSkipAccumulatorLoad(context);
 	} else if (pBest->flags) {
@@ -2122,7 +2118,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
 		context->is_aborted = true;
 		return;
 	}
-	if (sql_value_is_null(argv[0]))
+	if (mem_is_null(argv[0]))
 		return;
 	pAccum =
 	    (StrAccum *) sql_aggregate_context(context, sizeof(*pAccum));
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index ec6aaab64..abc9291ef 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -37,6 +37,142 @@
 #include "box/tuple.h"
 #include "mpstream/mpstream.h"
 
+bool
+mem_is_null(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Null) != 0;
+}
+
+bool
+mem_is_unsigned(const struct Mem *mem)
+{
+	return (mem->flags & MEM_UInt) != 0;
+}
+
+bool
+mem_is_string(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Str) != 0;
+}
+
+bool
+mem_is_number(const struct Mem *mem)
+{
+	return (mem->flags & (MEM_Real | MEM_Int |MEM_UInt)) != 0;
+}
+
+bool
+mem_is_double(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Real) != 0;
+}
+
+bool
+mem_is_integer(const struct Mem *mem)
+{
+	return (mem->flags & (MEM_Int | MEM_UInt)) != 0;
+}
+
+bool
+mem_is_boolean(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Bool) != 0;
+}
+
+bool
+mem_is_binary(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Blob) != 0;
+}
+
+bool
+mem_is_map(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Blob) != 0 &&
+	       (mem->flags & MEM_Subtype) != 0 &&
+	       mem->subtype == SQL_SUBTYPE_MSGPACK &&
+	       mp_typeof(*mem->z) == MP_MAP;
+}
+
+bool
+mem_is_array(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Blob) != 0 &&
+	       (mem->flags & MEM_Subtype) != 0 &&
+	       mem->subtype == SQL_SUBTYPE_MSGPACK &&
+	       mp_typeof(*mem->z) == MP_ARRAY;
+}
+
+bool
+mem_is_aggregate(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Agg) != 0;
+}
+
+bool
+mem_is_varstring(const struct Mem *mem)
+{
+	return (mem->flags & (MEM_Blob | MEM_Str)) != 0;
+}
+
+bool
+mem_is_frame(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Frame) != 0;
+}
+
+bool
+mem_is_undefined(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Undefined) != 0;
+}
+
+bool
+mem_is_static(const struct Mem *mem)
+{
+	return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+	       (mem->flags & MEM_Static) != 0;
+}
+
+bool
+mem_is_ephemeral(const struct Mem *mem)
+{
+	return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+	       (mem->flags & MEM_Ephem) != 0;
+}
+
+bool
+mem_is_dynamic(const struct Mem *mem)
+{
+	return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+	       (mem->flags & MEM_Dyn) != 0;
+}
+
+bool
+mem_is_allocated(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Blob) != 0 && mem->z == mem->zMalloc;
+}
+
+bool
+mem_is_cleared(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Null) != 0 && (mem->flags & MEM_Cleared) != 0;
+}
+
+bool
+mem_is_zeroblob(const struct Mem *mem)
+{
+	return (mem->flags & MEM_Blob) != 0 && (mem->flags & MEM_Zero) != 0;
+}
+
+bool
+mem_is_same_type(const struct Mem *mem1, const struct Mem *mem2)
+{
+	return (mem1->flags & MEM_PURE_TYPE_MASK) ==
+	       (mem2->flags & MEM_PURE_TYPE_MASK);
+}
+
 enum {
 	BUF_SIZE = 32,
 };
@@ -1083,8 +1219,7 @@ mem_convert_to_integer(struct Mem *mem)
 int
 mem_convert_to_numeric(struct Mem *mem, enum field_type type)
 {
-	assert(mp_type_is_numeric(mem_mp_type(mem)) &&
-	       sql_type_is_numeric(type));
+	assert(mem_is_number(mem) && sql_type_is_numeric(type));
 	assert(type != FIELD_TYPE_NUMBER);
 	if (type == FIELD_TYPE_DOUBLE)
 		return mem_convert_to_double(mem);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 5ac02c002..19920af3a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -87,6 +87,69 @@ struct Mem {
  */
 #define MEMCELLSIZE offsetof(Mem,zMalloc)
 
+bool
+mem_is_null(const struct Mem *mem);
+
+bool
+mem_is_unsigned(const struct Mem *mem);
+
+bool
+mem_is_string(const struct Mem *mem);
+
+bool
+mem_is_number(const struct Mem *mem);
+
+bool
+mem_is_double(const struct Mem *mem);
+
+bool
+mem_is_integer(const struct Mem *mem);
+
+bool
+mem_is_boolean(const struct Mem *mem);
+
+bool
+mem_is_binary(const struct Mem *mem);
+
+bool
+mem_is_map(const struct Mem *mem);
+
+bool
+mem_is_array(const struct Mem *mem);
+
+bool
+mem_is_aggregate(const struct Mem *mem);
+
+bool
+mem_is_varstring(const struct Mem *mem);
+
+bool
+mem_is_frame(const struct Mem *mem);
+
+bool
+mem_is_undefined(const struct Mem *mem);
+
+bool
+mem_is_static(const struct Mem *mem);
+
+bool
+mem_is_ephemeral(const struct Mem *mem);
+
+bool
+mem_is_dynamic(const struct Mem *mem);
+
+bool
+mem_is_allocated(const struct Mem *mem);
+
+bool
+mem_is_cleared(const struct Mem *mem);
+
+bool
+mem_is_zeroblob(const struct Mem *mem);
+
+bool
+mem_is_same_type(const struct Mem *mem1, const struct Mem *mem2);
+
 /**
  * Return a string that represent content of MEM. String is either allocated
  * using static_alloc() of just a static variable.
@@ -396,12 +459,6 @@ columnNullValue(void);
 
 /** Checkers. */
 
-static inline bool
-sql_value_is_null(struct Mem *value)
-{
-	return sql_value_type(value) == MP_NIL;
-}
-
 int sqlVdbeMemTooBig(Mem *);
 
 /* Return TRUE if Mem X contains dynamically allocated content - anything
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 0fa388ae9..b9107fccc 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1092,7 +1092,7 @@ selectInnerLoop(Parse * pParse,		/* The parser context */
 				 * re-use second for Null op-code.
 				 *
 				 * Change to an OP_Null sets the
-				 * MEM_Cleared bit on the first
+				 * Cleared flag on the first
 				 * register of the previous value. 
 				 * This will cause the OP_Ne below
 				 * to always fail on the first
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 12712efb4..05e0f78c1 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -91,8 +91,8 @@ int sql_sort_count = 0;
 #endif
 
 /*
- * The next global variable records the size of the largest MEM_Blob
- * or MEM_Str that has been used by a VDBE opcode.  The test procedures
+ * The next global variable records the size of the largest varbinary
+ * or string that has been used by a VDBE opcode.  The test procedures
  * use this information to make sure that the zero-blob functionality
  * is working correctly.   This variable has no function other than to
  * help verify the correct operation of the library.
@@ -102,9 +102,8 @@ int sql_max_blobsize = 0;
 static void
 updateMaxBlobsize(Mem *p)
 {
-	if ((p->flags & (MEM_Str|MEM_Blob))!=0 && p->n>sql_max_blobsize) {
+	if (mem_is_varstring(p) && p->n > sql_max_blobsize)
 		sql_max_blobsize = p->n;
-	}
 }
 #endif
 
@@ -396,8 +395,7 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
 	 * Add 0 termination (at most for strings)
 	 * Not sure why do we check MEM_Ephem
 	 */
-	if ((dest_mem->flags & (MEM_Ephem | MEM_Str)) ==
-	    (MEM_Ephem | MEM_Str)) {
+	if (mem_is_string(dest_mem) && mem_is_ephemeral(dest_mem)) {
 		int len = dest_mem->n;
 		if (dest_mem->szMalloc < len + 1) {
 			if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0)
@@ -630,7 +628,7 @@ case OP_Gosub: {            /* jump */
  */
 case OP_Return: {           /* in1 */
 	pIn1 = &aMem[pOp->p1];
-	assert(pIn1->flags==MEM_UInt);
+	assert(mem_is_unsigned(pIn1));
 	pOp = &aOp[pIn1->u.u];
 	pIn1->flags = MEM_Undefined;
 	break;
@@ -669,7 +667,7 @@ case OP_InitCoroutine: {     /* jump */
 case OP_EndCoroutine: {           /* in1 */
 	VdbeOp *pCaller;
 	pIn1 = &aMem[pOp->p1];
-	assert(pIn1->flags == MEM_UInt);
+	assert(mem_is_unsigned(pIn1));
 	assert(pIn1->u.u < (uint64_t) p->nOp);
 	pCaller = &aOp[pIn1->u.u];
 	assert(pCaller->opcode==OP_Yield);
@@ -1079,7 +1077,7 @@ case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
 	pIn2 = &aMem[pOp->p2];
 	pOut = vdbe_prepare_null_out(p, pOp->p3);
 	assert(pIn1!=pOut);
-	if ((pIn1->flags | pIn2->flags) & MEM_Null) {
+	if (mem_is_null(pIn1) || mem_is_null(pIn2)) {
 		/* Force NULL be of type STRING. */
 		pOut->field_type = FIELD_TYPE_STRING;
 		break;
@@ -1088,10 +1086,10 @@ case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
 	 * Concatenation operation can be applied only to
 	 * strings and blobs.
 	 */
-	uint32_t str_type_p1 = pIn1->flags & (MEM_Blob | MEM_Str);
-	uint32_t str_type_p2 = pIn2->flags & (MEM_Blob | MEM_Str);
-	if (str_type_p1 == 0 || str_type_p2 == 0) {
-		char *inconsistent_type = str_type_p1 == 0 ?
+	bool str_type_p1 = mem_is_varstring(pIn1);
+	bool str_type_p2 = mem_is_varstring(pIn2);
+	if (!str_type_p1 || !str_type_p2) {
+		char *inconsistent_type = !str_type_p1 ?
 					  mem_type_to_str(pIn1) :
 					  mem_type_to_str(pIn2);
 		diag_set(ClientError, ER_INCONSISTENT_TYPES,
@@ -1100,7 +1098,7 @@ case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
 	}
 
 	/* Moreover, both operands must be of the same type. */
-	if (str_type_p1 != str_type_p2) {
+	if (mem_is_string(pIn1) != mem_is_string(pIn2)) {
 		diag_set(ClientError, ER_INCONSISTENT_TYPES,
 			 mem_type_to_str(pIn2), mem_type_to_str(pIn1));
 		goto abort_due_to_error;
@@ -1114,7 +1112,7 @@ case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
 	if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) {
 		goto no_mem;
 	}
-	if (pIn1->flags & MEM_Str)
+	if (mem_is_string(pIn1))
 		MemSetTypeFlag(pOut, MEM_Str);
 	else
 		MemSetTypeFlag(pOut, MEM_Blob);
@@ -1173,7 +1171,6 @@ case OP_Subtract:              /* same as TK_MINUS, in1, in2, out3 */
 case OP_Multiply:              /* same as TK_STAR, in1, in2, out3 */
 case OP_Divide:                /* same as TK_SLASH, in1, in2, out3 */
 case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
-	u32 flags;      /* Combined MEM_* flags from both inputs */
 	u16 type1;      /* Numeric type of left operand */
 	u16 type2;      /* Numeric type of right operand */
 	i64 iA;         /* Integer value of left operand */
@@ -1186,14 +1183,16 @@ case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
 	pIn2 = &aMem[pOp->p2];
 	type2 = numericType(pIn2);
 	pOut = vdbe_prepare_null_out(p, pOp->p3);
-	flags = pIn1->flags | pIn2->flags;
-	if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
+	if (mem_is_null(pIn1) || mem_is_null(pIn2))
+		goto arithmetic_result_is_null;
 	if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
 	    (type2 & (MEM_Int | MEM_UInt)) != 0) {
 		iA = pIn1->u.i;
 		iB = pIn2->u.i;
-		bool is_lhs_neg = pIn1->flags & MEM_Int;
-		bool is_rhs_neg = pIn2->flags & MEM_Int;
+		bool is_lhs_neg = mem_is_integer(pIn1) &&
+				  !mem_is_unsigned(pIn1);
+		bool is_rhs_neg = mem_is_integer(pIn2) &&
+				  !mem_is_unsigned(pIn2);
 		bool is_res_neg;
 		switch( pOp->opcode) {
 		case OP_Add: {
@@ -1400,7 +1399,7 @@ case OP_BuiltinFunction: {
 		goto abort_due_to_error;
 
 	/* Copy the result of the function into register P3 */
-	if (pOut->flags & (MEM_Str|MEM_Blob)) {
+	if (mem_is_varstring(pOut)) {
 		if (sqlVdbeMemTooBig(pCtx->pOut)) goto too_big;
 	}
 
@@ -1459,7 +1458,7 @@ case OP_FunctionByName: {
 	 * Copy the result of the function invocation into
 	 * register P3.
 	 */
-	if ((pOut->flags & (MEM_Str | MEM_Blob)) != 0)
+	if (mem_is_varstring(pOut))
 		if (sqlVdbeMemTooBig(pOut)) goto too_big;
 
 	REGISTER_TRACE(p, pOp->p3, pOut);
@@ -1509,7 +1508,7 @@ case OP_ShiftRight: {           /* same as TK_RSHIFT, in1, in2, out3 */
 	pIn1 = &aMem[pOp->p1];
 	pIn2 = &aMem[pOp->p2];
 	pOut = vdbe_prepare_null_out(p, pOp->p3);
-	if ((pIn1->flags | pIn2->flags) & MEM_Null) {
+	if (mem_is_null(pIn1) || mem_is_null(pIn2)) {
 		/* Force NULL be of type INTEGER. */
 		pOut->field_type = FIELD_TYPE_INTEGER;
 		break;
@@ -1568,7 +1567,7 @@ case OP_ShiftRight: {           /* same as TK_RSHIFT, in1, in2, out3 */
 case OP_AddImm: {            /* in1 */
 	pIn1 = &aMem[pOp->p1];
 	memAboutToChange(p, pIn1);
-	assert((pIn1->flags & MEM_UInt) != 0 && pOp->p2 >= 0);
+	assert(mem_is_unsigned(pIn1) && pOp->p2 >= 0);
 	pIn1->u.u += pOp->p2;
 	break;
 }
@@ -1582,9 +1581,9 @@ case OP_AddImm: {            /* in1 */
  */
 case OP_MustBeInt: {            /* jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
+	if (!mem_is_integer(pIn1)) {
 		mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
-		if ((pIn1->flags & (MEM_Int | MEM_UInt)) == 0) {
+		if (!mem_is_integer(pIn1)) {
 			if (pOp->p2==0) {
 				diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 					 mem_str(pIn1), "integer");
@@ -1608,7 +1607,7 @@ case OP_MustBeInt: {            /* jump, in1 */
  */
 case OP_Realify: {                  /* in1 */
 	pIn1 = &aMem[pOp->p1];
-	if ((pIn1->flags & (MEM_Int | MEM_UInt)) != 0) {
+	if (mem_is_integer(pIn1)) {
 		sqlVdbeMemRealify(pIn1);
 	}
 	break;
@@ -1749,7 +1748,7 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 	flags3 = pIn3->flags;
 	enum field_type ft_p1 = pIn1->field_type;
 	enum field_type ft_p3 = pIn3->field_type;
-	if ((flags1 | flags3)&MEM_Null) {
+	if (mem_is_null(pIn1) || mem_is_null(pIn3)) {
 		/* One or both operands are NULL */
 		if (pOp->p5 & SQL_NULLEQ) {
 			/* If SQL_NULLEQ is set (which will only happen if the operator is
@@ -1757,11 +1756,10 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 			 * or not both operands are null.
 			 */
 			assert(pOp->opcode==OP_Eq || pOp->opcode==OP_Ne);
-			assert((flags1 & MEM_Cleared)==0);
+			assert(!mem_is_cleared(pIn1));
 			assert((pOp->p5 & SQL_JUMPIFNULL)==0);
-			if ((flags1&flags3&MEM_Null)!=0
-			    && (flags3&MEM_Cleared)==0
-				) {
+			if (mem_is_null(pIn1) && mem_is_null(pIn3) &&
+			    !mem_is_cleared(pIn3)) {
 				res = 0;  /* Operands are equal */
 			} else {
 				res = 1;  /* Operands are not equal */
@@ -1783,22 +1781,17 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 			}
 			break;
 		}
-	} else if (((flags1 | flags3) & MEM_Bool) != 0 ||
-		   ((flags1 | flags3) & MEM_Blob) != 0) {
-		/*
-		 * If one of values is of type BOOLEAN or VARBINARY,
-		 * then the second one must be of the same type as
-		 * well. Otherwise an error is raised.
-		 */
-		int type_arg1 = flags1 & (MEM_Bool | MEM_Blob);
-		int type_arg3 = flags3 & (MEM_Bool | MEM_Blob);
-		if (type_arg1 != type_arg3) {
-			char *inconsistent_type = type_arg1 != 0 ?
+	} else if (mem_is_boolean(pIn1) || mem_is_boolean(pIn3) ||
+		   mem_is_binary(pIn1) || mem_is_binary(pIn3)) {
+		if (!mem_is_same_type(pIn1, pIn3)) {
+			char *inconsistent_type = mem_is_boolean(pIn1) ||
+						  mem_is_binary(pIn1) ?
 						  mem_type_to_str(pIn3) :
 						  mem_type_to_str(pIn1);
-			char *expected_type     = type_arg1 != 0 ?
-						  mem_type_to_str(pIn1) :
-						  mem_type_to_str(pIn3);
+			char *expected_type = mem_is_boolean(pIn1) ||
+					      mem_is_binary(pIn1) ?
+					      mem_type_to_str(pIn1) :
+					      mem_type_to_str(pIn3);
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 				 inconsistent_type, expected_type);
 			goto abort_due_to_error;
@@ -1807,28 +1800,25 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 	} else {
 		enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
 		if (sql_type_is_numeric(type)) {
-			if ((flags1 | flags3)&MEM_Str) {
-				if ((flags1 & MEM_Str) == MEM_Str) {
-					mem_apply_numeric_type(pIn1);
-					testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
-					flags3 = pIn3->flags;
-				}
-				if ((flags3 & MEM_Str) == MEM_Str) {
-					if (mem_apply_numeric_type(pIn3) != 0) {
-						diag_set(ClientError,
-							 ER_SQL_TYPE_MISMATCH,
-							 mem_str(pIn3),
-							 "numeric");
-						goto abort_due_to_error;
-					}
-
+			if (mem_is_string(pIn1)) {
+				mem_apply_numeric_type(pIn1);
+				flags3 = pIn3->flags;
+			}
+			if (mem_is_string(pIn3)) {
+				if (mem_apply_numeric_type(pIn3) != 0) {
+					diag_set(ClientError,
+						 ER_SQL_TYPE_MISMATCH,
+						 mem_str(pIn3),
+						 "numeric");
+					goto abort_due_to_error;
 				}
 			}
 			/* Handle the common case of integer comparison here, as an
 			 * optimization, to avoid a call to sqlMemCompare()
 			 */
-			if ((pIn1->flags & pIn3->flags & (MEM_Int | MEM_UInt)) != 0) {
-				if ((pIn1->flags & pIn3->flags & MEM_Int) != 0) {
+			if (mem_is_integer(pIn1) && mem_is_integer(pIn3)) {
+				if (!mem_is_unsigned(pIn1) &&
+				    !mem_is_unsigned(pIn3)) {
 					if (pIn3->u.i > pIn1->u.i)
 						res = +1;
 					else if (pIn3->u.i < pIn1->u.i)
@@ -1837,7 +1827,8 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 						res = 0;
 					goto compare_op;
 				}
-				if ((pIn1->flags & pIn3->flags & MEM_UInt) != 0) {
+				if (mem_is_unsigned(pIn1) &&
+				    mem_is_unsigned(pIn3)) {
 					if (pIn3->u.u > pIn1->u.u)
 						res = +1;
 					else if (pIn3->u.u < pIn1->u.u)
@@ -1846,8 +1837,8 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 						res = 0;
 					goto compare_op;
 				}
-				if ((pIn1->flags & MEM_UInt) != 0 &&
-				    (pIn3->flags & MEM_Int) != 0) {
+				if (mem_is_unsigned(pIn1) &&
+				    !mem_is_unsigned(pIn3)) {
 					res = -1;
 					goto compare_op;
 				}
@@ -1855,21 +1846,13 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 				goto compare_op;
 			}
 		} else if (type == FIELD_TYPE_STRING) {
-			if ((flags1 & MEM_Str) == 0 &&
-			    (flags1 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
-				testcase( pIn1->flags & MEM_Int);
-				testcase( pIn1->flags & MEM_Real);
+			if (!mem_is_string(pIn1) && mem_is_number(pIn1)) {
 				sqlVdbeMemStringify(pIn1);
-				testcase( (flags1&MEM_Dyn) != (pIn1->flags&MEM_Dyn));
 				flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
 				assert(pIn1!=pIn3);
 			}
-			if ((flags3 & MEM_Str) == 0 &&
-			    (flags3 & (MEM_Int | MEM_UInt | MEM_Real)) != 0) {
-				testcase( pIn3->flags & MEM_Int);
-				testcase( pIn3->flags & MEM_Real);
+			if (!mem_is_string(pIn3) && mem_is_number(pIn3)) {
 				sqlVdbeMemStringify(pIn3);
-				testcase( (flags3&MEM_Dyn) != (pIn3->flags&MEM_Dyn));
 				flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
 			}
 		}
@@ -2077,9 +2060,9 @@ case OP_Or: {             /* same as TK_OR, in1, in2, out3 */
 	int v2;    /* Right operand: 0==FALSE, 1==TRUE, 2==UNKNOWN or NULL */
 
 	pIn1 = &aMem[pOp->p1];
-	if (pIn1->flags & MEM_Null) {
+	if (mem_is_null(pIn1)) {
 		v1 = 2;
-	} else if ((pIn1->flags & MEM_Bool) != 0) {
+	} else if (mem_is_boolean(pIn1)) {
 		v1 = pIn1->u.b;
 	} else {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2087,9 +2070,9 @@ case OP_Or: {             /* same as TK_OR, in1, in2, out3 */
 		goto abort_due_to_error;
 	}
 	pIn2 = &aMem[pOp->p2];
-	if (pIn2->flags & MEM_Null) {
+	if (mem_is_null(pIn2)) {
 		v2 = 2;
-	} else if ((pIn2->flags & MEM_Bool) != 0) {
+	} else if (mem_is_boolean(pIn2)) {
 		v2 = pIn2->u.b;
 	} else {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2120,8 +2103,8 @@ case OP_Not: {                /* same as TK_NOT, in1, out2 */
 	pIn1 = &aMem[pOp->p1];
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	pOut->field_type = FIELD_TYPE_BOOLEAN;
-	if ((pIn1->flags & MEM_Null)==0) {
-		if ((pIn1->flags & MEM_Bool) == 0) {
+	if (!mem_is_null(pIn1)) {
+		if (!mem_is_boolean(pIn1)) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 				 mem_str(pIn1), "boolean");
 			goto abort_due_to_error;
@@ -2143,7 +2126,7 @@ case OP_BitNot: {             /* same as TK_BITNOT, in1, out2 */
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	/* Force NULL be of type INTEGER. */
 	pOut->field_type = FIELD_TYPE_INTEGER;
-	if ((pIn1->flags & MEM_Null)==0) {
+	if (!mem_is_null(pIn1)) {
 		int64_t i;
 		bool is_neg;
 		if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
@@ -2188,9 +2171,9 @@ case OP_If:                 /* jump, in1 */
 case OP_IfNot: {            /* jump, in1 */
 	int c;
 	pIn1 = &aMem[pOp->p1];
-	if (pIn1->flags & MEM_Null) {
+	if (mem_is_null(pIn1)) {
 		c = pOp->p3;
-	} else if ((pIn1->flags & MEM_Bool) != 0) {
+	} else if (mem_is_boolean(pIn1)) {
 		c = pOp->opcode == OP_IfNot ? ! pIn1->u.b : pIn1->u.b;
 	} else {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -2211,8 +2194,8 @@ case OP_IfNot: {            /* jump, in1 */
  */
 case OP_IsNull: {            /* same as TK_ISNULL, jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	VdbeBranchTaken( (pIn1->flags & MEM_Null)!=0, 2);
-	if ((pIn1->flags & MEM_Null)!=0) {
+	VdbeBranchTaken(mem_is_null(pIn1), 2);
+	if (mem_is_null(pIn1)) {
 		goto jump_to_p2;
 	}
 	break;
@@ -2225,8 +2208,8 @@ case OP_IsNull: {            /* same as TK_ISNULL, jump, in1 */
  */
 case OP_NotNull: {            /* same as TK_NOTNULL, jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	VdbeBranchTaken( (pIn1->flags & MEM_Null)==0, 2);
-	if ((pIn1->flags & MEM_Null)==0) {
+	VdbeBranchTaken(!mem_is_null(pIn1), 2);
+	if (!mem_is_null(pIn1)) {
 		goto jump_to_p2;
 	}
 	break;
@@ -2280,7 +2263,7 @@ case OP_Column: {
 			if (pC->eCurType==CURTYPE_PSEUDO) {
 				assert(pC->uc.pseudoTableReg>0);
 				pReg = &aMem[pC->uc.pseudoTableReg];
-				assert(pReg->flags & MEM_Blob);
+				assert(mem_is_binary(pReg));
 				assert(memIsValid(pReg));
 				vdbe_field_ref_prepare_data(&pC->field_ref,
 							    pReg->z, pReg->n);
@@ -2309,7 +2292,7 @@ case OP_Column: {
 	if (vdbe_field_ref_fetch(&pC->field_ref, p2, pDest) != 0)
 		goto abort_due_to_error;
 
-	if ((pDest->flags & MEM_Null) &&
+	if (mem_is_null(pDest) &&
 	    (uint32_t) p2  >= pC->field_ref.field_count &&
 	    default_val_mem != NULL) {
 		sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static);
@@ -2364,7 +2347,7 @@ case OP_ApplyType: {
 			if (!sql_type_is_numeric(type))
 				goto type_mismatch;
 			/* Implicit cast is allowed only from numeric type. */
-			if (!mp_type_is_numeric(mem_mp_type(pIn1)))
+			if (!mem_is_number(pIn1))
 				goto type_mismatch;
 			/* Try to convert numeric-to-numeric. */
 			if (mem_convert_to_numeric(pIn1, type) != 0)
@@ -2982,18 +2965,18 @@ case OP_SeekGT: {       /* jump, in3 */
 		 * the seek, so convert it.
 		 */
 		pIn3 = &aMem[int_field];
-		if ((pIn3->flags & MEM_Null) != 0)
+		if (mem_is_null(pIn3))
 			goto skip_truncate;
-		if ((pIn3->flags & MEM_Str) != 0)
+		if (mem_is_string(pIn3))
 			mem_apply_numeric_type(pIn3);
 		int64_t i;
-		if ((pIn3->flags & MEM_Int) == MEM_Int) {
+		if (mem_is_integer(pIn3) && !mem_is_unsigned(pIn3)) {
 			i = pIn3->u.i;
 			is_neg = true;
-		} else if ((pIn3->flags & MEM_UInt) == MEM_UInt) {
+		} else if (mem_is_unsigned(pIn3)) {
 			i = pIn3->u.u;
 			is_neg = false;
-		} else if ((pIn3->flags & MEM_Real) == MEM_Real) {
+		} else if (mem_is_double(pIn3)) {
 			if (pIn3->u.r > (double)INT64_MAX)
 				i = INT64_MAX;
 			else if (pIn3->u.r < (double)INT64_MIN)
@@ -3011,8 +2994,8 @@ case OP_SeekGT: {       /* jump, in3 */
 		/* If the P3 value could not be converted into an integer without
 		 * loss of information, then special processing is required...
 		 */
-		if ((pIn3->flags & (MEM_Int | MEM_UInt)) == 0) {
-			if ((pIn3->flags & MEM_Real)==0) {
+		if (!mem_is_integer(pIn3)) {
+			if (!mem_is_double(pIn3)) {
 				/* If the P3 value cannot be converted into any kind of a number,
 				 * then the seek is not possible, so jump to P2
 				 */
@@ -3218,7 +3201,8 @@ case OP_Found: {        /* jump, in3 */
 #ifdef SQL_DEBUG
 		for(ii=0; ii<r.nField; ii++) {
 			assert(memIsValid(&r.aMem[ii]));
-			assert((r.aMem[ii].flags & MEM_Zero)==0 || r.aMem[ii].n==0);
+			assert(!mem_is_zeroblob(&r.aMem[ii]) ||
+			       r.aMem[ii].n == 0);
 			if (ii != 0)
 				REGISTER_TRACE(p, pOp->p3+ii, &r.aMem[ii]);
 		}
@@ -3228,7 +3212,7 @@ case OP_Found: {        /* jump, in3 */
 	} else {
 		pFree = pIdxKey = sqlVdbeAllocUnpackedRecord(db, pC->key_def);
 		if (pIdxKey==0) goto no_mem;
-		assert(pIn3->flags & MEM_Blob );
+		assert(mem_is_binary(pIn3));
 		(void)ExpandBlob(pIn3);
 		sqlVdbeRecordUnpackMsgpack(pC->key_def,
 					       pIn3->z, pIdxKey);
@@ -3242,7 +3226,7 @@ case OP_Found: {        /* jump, in3 */
 		 * conflict
 		 */
 		for(ii=0; ii<pIdxKey->nField; ii++) {
-			if (pIdxKey->aMem[ii].flags & MEM_Null) {
+			if (mem_is_null(&pIdxKey->aMem[ii])) {
 				takeJump = 1;
 				break;
 			}
@@ -3352,11 +3336,11 @@ case OP_FCopy: {     /* out2 */
 		pIn1 = &aMem[pOp->p1];
 	}
 
-	if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && (pIn1->flags & MEM_Null)) {
+	if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && mem_is_null(pIn1)) {
 		pOut = vdbe_prepare_null_out(p, pOp->p2);
 	} else {
 		assert(memIsValid(pIn1));
-		assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
+		assert(mem_is_integer(pIn1));
 
 		pOut = vdbe_prepare_null_out(p, pOp->p2);
 		mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
@@ -3491,7 +3475,7 @@ case OP_SorterData: {
 	assert(isSorter(pC));
 	if (sqlVdbeSorterRowkey(pC, pOut) != 0)
 		goto abort_due_to_error;
-	assert(pOut->flags & MEM_Blob);
+	assert(mem_is_binary(pOut));
 	assert(pOp->p1>=0 && pOp->p1<p->nCursor);
 	p->apCsr[pOp->p3]->cacheStatus = CACHE_STALE;
 	break;
@@ -3849,7 +3833,7 @@ case OP_SorterInsert: {      /* in2 */
 	assert(cursor != NULL);
 	assert(isSorter(cursor));
 	pIn2 = &aMem[pOp->p2];
-	assert((pIn2->flags & MEM_Blob) != 0);
+	assert(mem_is_binary(pIn2));
 	if (ExpandBlob(pIn2) != 0 ||
 	    sqlVdbeSorterWrite(cursor, pIn2) != 0)
 		goto abort_due_to_error;
@@ -3883,7 +3867,7 @@ case OP_SorterInsert: {      /* in2 */
 case OP_IdxReplace:
 case OP_IdxInsert: {
 	pIn2 = &aMem[pOp->p1];
-	assert((pIn2->flags & MEM_Blob) != 0);
+	assert(mem_is_binary(pIn2));
 	if (ExpandBlob(pIn2) != 0)
 		goto abort_due_to_error;
 	struct space *space;
@@ -3894,7 +3878,7 @@ case OP_IdxInsert: {
 	assert(space != NULL);
 	if (space->def->id != 0) {
 		/* Make sure that memory has been allocated on region. */
-		assert(aMem[pOp->p1].flags & MEM_Ephem);
+		assert(mem_is_ephemeral(&aMem[pOp->p1]));
 		if (pOp->opcode == OP_IdxInsert) {
 			rc = tarantoolsqlInsert(space, pIn2->z,
 						    pIn2->z + pIn2->n);
@@ -3928,7 +3912,7 @@ case OP_IdxInsert: {
 	}
 	if ((pOp->p5 & OPFLAG_NCHANGE) != 0)
 		p->nChange++;
-	if (pOp->p3 > 0 && ((aMem[pOp->p3].flags) & MEM_Null) != 0) {
+	if (pOp->p3 > 0 && mem_is_null(&aMem[pOp->p3])) {
 		assert(space->sequence != NULL);
 		int64_t value;
 		if (sequence_get_value(space->sequence, &value) != 0)
@@ -3974,10 +3958,10 @@ case OP_Update: {
 	assert(pOp->p4type == P4_SPACEPTR);
 
 	struct Mem *key_mem = &aMem[pOp->p2];
-	assert((key_mem->flags & MEM_Blob) != 0);
+	assert(mem_is_binary(key_mem));
 
 	struct Mem *upd_fields_mem = &aMem[pOp->p3];
-	assert((upd_fields_mem->flags & MEM_Blob) != 0);
+	assert(mem_is_binary(upd_fields_mem));
 	uint32_t *upd_fields = (uint32_t *)upd_fields_mem->z;
 	uint32_t upd_fields_cnt = upd_fields_mem->n / sizeof(uint32_t);
 
@@ -4409,7 +4393,7 @@ case OP_Program: {        /* jump */
 	 * the trigger program. If this trigger has been fired before, then pRt
 	 * is already allocated. Otherwise, it must be initialized.
 	 */
-	if ((pRt->flags&MEM_Frame)==0) {
+	if (!mem_is_frame(pRt)) {
 		/* SubProgram.nMem is set to the number of memory cells used by the
 		 * program stored in SubProgram.aOp. As well as these, one memory
 		 * cell is required for each cursor used by the program. Set local
@@ -4549,8 +4533,8 @@ case OP_FkIfZero: {         /* jump */
  */
 case OP_IfPos: {        /* jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
-	if ((pIn1->flags & MEM_UInt) != 0 && pIn1->u.u != 0) {
+	assert(mem_is_integer(pIn1));
+	if (mem_is_unsigned(pIn1) && pIn1->u.u != 0) {
 		assert(pOp->p3 >= 0);
 		uint64_t res = pIn1->u.u - (uint64_t) pOp->p3;
 		/*
@@ -4584,8 +4568,8 @@ case OP_OffsetLimit: {    /* in1, out2, in3 */
 	pIn3 = &aMem[pOp->p3];
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 
-	assert((pIn1->flags & MEM_UInt) != 0);
-	assert((pIn3->flags & MEM_UInt) != 0);
+	assert(mem_is_unsigned(pIn1));
+	assert(mem_is_unsigned(pIn3));
 	uint64_t x = pIn1->u.u;
 	uint64_t rhs = pIn3->u.u;
 	bool unused;
@@ -4608,7 +4592,7 @@ case OP_OffsetLimit: {    /* in1, out2, in3 */
  */
 case OP_IfNotZero: {        /* jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	assert((pIn1->flags & MEM_UInt) != 0);
+	assert(mem_is_unsigned(pIn1));
 	if (pIn1->u.u > 0) {
 		pIn1->u.u--;
 		goto jump_to_p2;
@@ -4624,7 +4608,7 @@ case OP_IfNotZero: {        /* jump, in1 */
  */
 case OP_DecrJumpZero: {      /* jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	assert((pIn1->flags & MEM_UInt) != 0);
+	assert(mem_is_unsigned(pIn1));
 	if (pIn1->u.u > 0)
 		pIn1->u.u--;
 	if (pIn1->u.u == 0) goto jump_to_p2;
@@ -4721,7 +4705,7 @@ case OP_AggStep: {
 		mem_destroy(&t);
 		goto abort_due_to_error;
 	}
-	assert(t.flags==MEM_Null);
+	assert(mem_is_null(&t));
 	if (pCtx->skipFlag) {
 		assert(pOp[-1].opcode==OP_CollSeq);
 		i = pOp[-1].p1;
@@ -4747,7 +4731,7 @@ case OP_AggFinal: {
 	Mem *pMem;
 	assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
 	pMem = &aMem[pOp->p1];
-	assert((pMem->flags & ~(MEM_Null|MEM_Agg))==0);
+	assert(mem_is_null(pMem) || mem_is_aggregate(pMem));
 	if (sql_vdbemem_finalize(pMem, pOp->p4.func) != 0)
 		goto abort_due_to_error;
 	UPDATE_MAX_BLOBSIZE(pMem);
@@ -4874,7 +4858,7 @@ case OP_SetSession: {
 	struct session_setting *setting = &session_settings[sid];
 	switch (setting->field_type) {
 	case FIELD_TYPE_BOOLEAN: {
-		if ((pIn1->flags & MEM_Bool) == 0)
+		if (!mem_is_boolean(pIn1))
 			goto invalid_type;
 		bool value = pIn1->u.b;
 		size_t size = mp_sizeof_bool(value);
@@ -4885,7 +4869,7 @@ case OP_SetSession: {
 		break;
 	}
 	case FIELD_TYPE_STRING: {
-		if ((pIn1->flags & MEM_Str) == 0)
+		if (!mem_is_string(pIn1))
 			goto invalid_type;
 		const char *str = pIn1->z;
 		uint32_t size = mp_sizeof_str(pIn1->n);
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 338b8bc96..9615c36a6 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -333,17 +333,6 @@ int sqlVdbeList(Vdbe *);
 
 int sqlVdbeHalt(Vdbe *);
 
-/**
- * In terms of VDBE memory cell type, _BIN, _ARRAY and _MAP
- * messagepacks are stored as binary string (i.e. featuring
- * MEM_Blob internal type).
- */
-#define mp_type_is_bloblike(X) ((X) == MP_BIN || (X) == MP_ARRAY || (X) == MP_MAP)
-
-/** Return TRUE if MP_type of X is numeric, FALSE otherwise. */
-#define mp_type_is_numeric(X) ((X) == MP_INT || (X) == MP_UINT ||\
-			       (X) == MP_DOUBLE)
-
 const char *sqlOpcodeName(int);
 int sqlVdbeCloseStatement(Vdbe *, int);
 void sqlVdbeFrameDelete(VdbeFrame *);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index a195f8dfd..8a9573c57 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -380,7 +380,7 @@ static SQL_NOINLINE void *
 createAggContext(sql_context * p, int nByte)
 {
 	Mem *pMem = p->pMem;
-	assert((pMem->flags & MEM_Agg) == 0);
+	assert(!mem_is_aggregate(pMem));
 	if (nByte <= 0) {
 		sqlVdbeMemSetNull(pMem);
 		pMem->z = 0;
@@ -407,7 +407,7 @@ sql_aggregate_context(sql_context * p, int nByte)
 	assert(p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
 	assert(p->func->def->aggregate == FUNC_AGGREGATE_GROUP);
 	testcase(nByte < 0);
-	if ((p->pMem->flags & MEM_Agg) == 0) {
+	if (!mem_is_aggregate(p->pMem)) {
 		return createAggContext(p, nByte);
 	} else {
 		return (void *)p->pMem->z;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 55d4fc14f..11304b5bc 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1253,7 +1253,7 @@ sqlVdbeList(Vdbe * p)
 		 */
 		assert(p->nMem > 9);
 		pSub = &p->aMem[9];
-		if (pSub->flags & MEM_Blob) {
+		if (mem_is_binary(pSub)) {
 			/* On the first call to sql_step(), pSub will hold a NULL.  It is
 			 * initialized to a BLOB by the P4_SUBPROGRAM processing logic below
 			 */
@@ -1698,7 +1698,7 @@ Cleanup(Vdbe * p)
 			assert(p->apCsr[i] == 0);
 	if (p->aMem) {
 		for (i = 0; i < p->nMem; i++)
-			assert(p->aMem[i].flags == MEM_Undefined);
+			assert(mem_is_undefined(&p->aMem[i]));
 	}
 #endif
 
@@ -2306,7 +2306,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
 	assert(iVar > 0);
 	if (v) {
 		Mem *pMem = &v->aVar[iVar - 1];
-		if (0 == (pMem->flags & MEM_Null)) {
+		if (!mem_is_null(pMem)) {
 			sql_value *pRet = sqlValueNew(v->db);
 			if (pRet) {
 				sqlVdbeMemCopy((Mem *) pRet, pMem);
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index 927f85559..a9a5f45af 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -2218,7 +2218,7 @@ sqlVdbeSorterCompare(const VdbeCursor * pCsr,	/* Sorter cursor */
 	pKey = vdbeSorterRowkey(pSorter, &nKey);
 	sqlVdbeRecordUnpackMsgpack(pCsr->key_def, pKey, r2);
 	for (i = 0; i < nKeyCol; i++) {
-		if (r2->aMem[i].flags & MEM_Null) {
+		if (mem_is_null(&r2->aMem[i])) {
 			*pRes = -1;
 			return 0;
 		}
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 0c002dbee..782e44734 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -312,9 +312,8 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
 		pVal =
 		    sqlVdbeGetBoundValue(pReprepare, iCol,
 					     FIELD_TYPE_SCALAR);
-		if (pVal && sql_value_type(pVal) == MP_STR) {
+		if (pVal != NULL && mem_is_string(pVal))
 			z = (char *)sql_value_text(pVal);
-		}
 		assert(pRight->op == TK_VARIABLE || pRight->op == TK_REGISTER);
 	} else if (op == TK_STRING) {
 		z = pRight->u.zToken;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 12/53] sql: introduce mem_copy()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (10 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 11/53] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 13/53] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
                   ` (40 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_copy(). This function copies value from source
MEM to destination MEM. In case value is string or binary and have not
static allocation type, it is copied to newly allocated memory.

Part of #5818
---
 src/box/sql/func.c    |  4 ++--
 src/box/sql/mem.c     | 54 +++++++++++++++++++++++++------------------
 src/box/sql/mem.h     |  9 +++++++-
 src/box/sql/vdbeapi.c |  2 +-
 src/box/sql/vdbeaux.c |  2 +-
 5 files changed, 44 insertions(+), 27 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 81b537d9b..6b6081150 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -2079,13 +2079,13 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
 		bool is_max = (func->flags & SQL_FUNC_MAX) != 0;
 		cmp = sqlMemCompare(pBest, pArg, pColl);
 		if ((is_max && cmp < 0) || (!is_max && cmp > 0)) {
-			sqlVdbeMemCopy(pBest, pArg);
+			mem_copy(pBest, pArg);
 		} else {
 			sqlSkipAccumulatorLoad(context);
 		}
 	} else {
 		pBest->db = sql_context_db_handle(context);
-		sqlVdbeMemCopy(pBest, pArg);
+		mem_copy(pBest, pArg);
 	}
 }
 
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index abc9291ef..f12441d7c 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -257,6 +257,38 @@ mem_destroy(struct Mem *mem)
 	mem->zMalloc = NULL;
 }
 
+int
+mem_copy(struct Mem *to, const struct Mem *from)
+{
+	mem_clear(to);
+	to->u = from->u;
+	to->flags = from->flags;
+	to->subtype = from->subtype;
+	to->field_type = from->field_type;
+	to->n = from->n;
+	to->z = from->z;
+	if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
+		return 0;
+	if ((to->flags & MEM_Static) != 0)
+		return 0;
+	if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob)) {
+		if (sqlVdbeMemExpandBlob(to) != 0)
+			return -1;
+		return 0;
+	}
+	if (to->szMalloc == 0)
+		to->zMalloc = sqlDbMallocRaw(to->db, to->n);
+	else
+		to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
+	if (to->zMalloc == NULL)
+		return -1;
+	to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
+	memcpy(to->zMalloc, to->z, to->n);
+	to->z = to->zMalloc;
+	to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
+	return 0;
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -1993,28 +2025,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
 	return 0;
 }
 
-/*
- * Make a full copy of pFrom into pTo.  Prior contents of pTo are
- * freed before the copy is made.
- */
-int
-sqlVdbeMemCopy(Mem * pTo, const Mem * pFrom)
-{
-	int rc = 0;
-
-	mem_clear(pTo);
-	memcpy(pTo, pFrom, MEMCELLSIZE);
-	pTo->flags &= ~MEM_Dyn;
-	if (pTo->flags & (MEM_Str | MEM_Blob)) {
-		if (0 == (pFrom->flags & MEM_Static)) {
-			pTo->flags |= MEM_Ephem;
-			rc = sqlVdbeMemMakeWriteable(pTo);
-		}
-	}
-
-	return rc;
-}
-
 void
 sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 19920af3a..fb649891a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -169,6 +169,14 @@ mem_create(struct Mem *mem);
 void
 mem_destroy(struct Mem *mem);
 
+/**
+ * Copy content of MEM from one MEM to another. In case source MEM contains
+ * string or binary and allocation type is not STATIC, this value is copied to
+ * newly allocated by destination MEM memory.
+ */
+int
+mem_copy(struct Mem *to, const struct Mem *from);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -485,7 +493,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
 
 int
 vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-int sqlVdbeMemCopy(Mem *, const Mem *);
 void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
 void sqlVdbeMemMove(Mem *, Mem *);
 int sqlVdbeMemMakeWriteable(Mem *);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 8a9573c57..b13385647 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -229,7 +229,7 @@ sql_result_text64(sql_context * pCtx,
 void
 sql_result_value(sql_context * pCtx, sql_value * pValue)
 {
-	sqlVdbeMemCopy(pCtx->pOut, pValue);
+	mem_copy(pCtx->pOut, pValue);
 }
 
 void
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 11304b5bc..21ac84099 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2309,7 +2309,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
 		if (!mem_is_null(pMem)) {
 			sql_value *pRet = sqlValueNew(v->db);
 			if (pRet) {
-				sqlVdbeMemCopy((Mem *) pRet, pMem);
+				mem_copy(pRet, pMem);
 				sql_value_apply_type(pRet, aff);
 			}
 			return pRet;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 13/53] sql: introduce mem_copy_as_ephemeral()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (11 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 12/53] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 14/53] sql: rework mem_move() Mergen Imeev via Tarantool-patches
                   ` (39 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch intoduces mem_copy_as_ephemeral(). This function copies value
from source MEM to destination MEM. In case value is of string or binary
type and its allocation type is not static, it copied as value with
ephemeral allocation type.

Part of #5818
---
 src/box/sql/mem.c  | 58 ++++++++++++++++++++--------------------------
 src/box/sql/mem.h  | 15 ++++++------
 src/box/sql/vdbe.c | 10 ++++----
 3 files changed, 38 insertions(+), 45 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index f12441d7c..30c568970 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -289,6 +289,25 @@ mem_copy(struct Mem *to, const struct Mem *from)
 	return 0;
 }
 
+int
+mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
+{
+	mem_clear(to);
+	to->u = from->u;
+	to->flags = from->flags;
+	to->subtype = from->subtype;
+	to->field_type = from->field_type;
+	to->n = from->n;
+	to->z = from->z;
+	if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
+		return 0;
+	if ((to->flags & (MEM_Static | MEM_Ephem)) != 0)
+		return 0;
+	to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Zero | MEM_Subtype);
+	to->flags |= MEM_Ephem;
+	return 0;
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -400,20 +419,6 @@ vdbeMemAddTerminator(Mem * pMem)
 	return 0;
 }
 
-/*
- * Make an shallow copy of pFrom into pTo.  Prior contents of
- * pTo are freed.  The pFrom->z field is not duplicated.  If
- * pFrom->z is used, then pTo->z points to the same thing as pFrom->z
- * and flags gets srcType (either MEM_Ephem or MEM_Static).
- */
-static SQL_NOINLINE void
-vdbeClrCopy(Mem * pTo, const Mem * pFrom, int eType)
-{
-	mem_clear(pTo);
-	assert(!VdbeMemDynamic(pTo));
-	sqlVdbeMemShallowCopy(pTo, pFrom, eType);
-}
-
 /*
  * Both *pMem1 and *pMem2 contain string values. Compare the two values
  * using the collation sequence pColl. As usual, return a negative , zero
@@ -593,9 +598,12 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
 	int i;
 	Mem *pX;
 	for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
-		if (pX->pScopyFrom == pMem) {
-			pX->flags |= MEM_Undefined;
-			pX->pScopyFrom = 0;
+		if ((pX->flags & (MEM_Blob | MEM_Str)) != 0 &&
+		    (pX->flags & (MEM_Ephem | MEM_Static)) == 0) {
+			if (pX->pScopyFrom == pMem) {
+				pX->flags |= MEM_Undefined;
+				pX->pScopyFrom = 0;
+			}
 		}
 	}
 	pMem->pScopyFrom = 0;
@@ -2025,22 +2033,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
 	return 0;
 }
 
-void
-sqlVdbeMemShallowCopy(Mem * pTo, const Mem * pFrom, int srcType)
-{
-	assert(pTo->db == pFrom->db);
-	if (VdbeMemDynamic(pTo)) {
-		vdbeClrCopy(pTo, pFrom, srcType);
-		return;
-	}
-	memcpy(pTo, pFrom, MEMCELLSIZE);
-	if ((pFrom->flags & MEM_Static) == 0) {
-		pTo->flags &= ~(MEM_Dyn | MEM_Static | MEM_Ephem);
-		assert(srcType == MEM_Ephem || srcType == MEM_Static);
-		pTo->flags |= srcType;
-	}
-}
-
 /*
  * Transfer the contents of pFrom to pTo. Any existing value in pTo is
  * freed. If pFrom contains ephemeral data, a copy is made.
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index fb649891a..58220d215 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -81,12 +81,6 @@ struct Mem {
 #endif
 };
 
-/*
- * Size of struct Mem not including the Mem.zMalloc member or anything that
- * follows.
- */
-#define MEMCELLSIZE offsetof(Mem,zMalloc)
-
 bool
 mem_is_null(const struct Mem *mem);
 
@@ -177,6 +171,14 @@ mem_destroy(struct Mem *mem);
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
+/**
+ * Copy content of MEM from one MEM to another. In case source MEM contains
+ * string or binary and allocation type is not STATIC, this value is copied as
+ * value with ephemeral allocation type.
+ */
+int
+mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -493,7 +495,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
 
 int
 vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-void sqlVdbeMemShallowCopy(Mem *, const Mem *, int);
 void sqlVdbeMemMove(Mem *, Mem *);
 int sqlVdbeMemMakeWriteable(Mem *);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 05e0f78c1..55083fb23 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -917,7 +917,7 @@ case OP_Variable: {            /* out2 */
 		goto too_big;
 	}
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	sqlVdbeMemShallowCopy(pOut, pVar, MEM_Static);
+	mem_copy_as_ephemeral(pOut, pVar);
 	UPDATE_MAX_BLOBSIZE(pOut);
 	break;
 }
@@ -979,7 +979,7 @@ case OP_Copy: {
 	pOut = &aMem[pOp->p2];
 	assert(pOut!=pIn1);
 	while( 1) {
-		sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
+		mem_copy_as_ephemeral(pOut, pIn1);
 		Deephemeralize(pOut);
 #ifdef SQL_DEBUG
 		pOut->pScopyFrom = 0;
@@ -1009,7 +1009,7 @@ case OP_SCopy: {            /* out2 */
 	pIn1 = &aMem[pOp->p1];
 	pOut = &aMem[pOp->p2];
 	assert(pOut!=pIn1);
-	sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
+	mem_copy_as_ephemeral(pOut, pIn1);
 #ifdef SQL_DEBUG
 	if (pOut->pScopyFrom==0) pOut->pScopyFrom = pIn1;
 #endif
@@ -2295,7 +2295,7 @@ case OP_Column: {
 	if (mem_is_null(pDest) &&
 	    (uint32_t) p2  >= pC->field_ref.field_count &&
 	    default_val_mem != NULL) {
-		sqlVdbeMemShallowCopy(pDest, default_val_mem, MEM_Static);
+		mem_copy_as_ephemeral(pDest, default_val_mem);
 	}
 	pDest->field_type = field_type;
 op_column_out:
@@ -4473,7 +4473,7 @@ case OP_Param: {           /* out2 */
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	pFrame = p->pFrame;
 	pIn = &pFrame->aMem[pOp->p1 + pFrame->aOp[pFrame->pc].p1];
-	sqlVdbeMemShallowCopy(pOut, pIn, MEM_Ephem);
+	mem_copy_as_ephemeral(pOut, pIn);
 	break;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 14/53] sql: rework mem_move()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (12 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 13/53] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 15/53] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
                   ` (38 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch reworks mem_move(). This function moves all content of source
MEM to destination MEM. Source mem is set to NULL.

Part of #5818
---
 src/box/sql/mem.c     | 57 +++++++++----------------------------------
 src/box/sql/mem.h     |  8 ++++--
 src/box/sql/vdbe.c    | 29 ++--------------------
 src/box/sql/vdbeapi.c |  2 +-
 4 files changed, 20 insertions(+), 76 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 30c568970..3d42ac63c 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -308,6 +308,17 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)
 	return 0;
 }
 
+int
+mem_move(struct Mem *to, struct Mem *from)
+{
+	mem_destroy(to);
+	memcpy(to, from, sizeof(*to));
+	from->flags = MEM_Null;
+	from->szMalloc = 0;
+	from->zMalloc = NULL;
+	return 0;
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -2033,52 +2044,6 @@ vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
 	return 0;
 }
 
-/*
- * Transfer the contents of pFrom to pTo. Any existing value in pTo is
- * freed. If pFrom contains ephemeral data, a copy is made.
- *
- * pFrom contains an SQL NULL when this routine returns.
- */
-void
-sqlVdbeMemMove(Mem * pTo, Mem * pFrom)
-{
-	assert(pFrom->db == 0 || pTo->db == 0 || pFrom->db == pTo->db);
-
-	mem_destroy(pTo);
-	memcpy(pTo, pFrom, sizeof(Mem));
-	pFrom->flags = MEM_Null;
-	pFrom->szMalloc = 0;
-}
-
-/*
- * Change pMem so that its MEM_Str or MEM_Blob value is stored in
- * MEM.zMalloc, where it can be safely written.
- *
- * Return 0 on success or -1 if malloc fails.
- */
-int
-sqlVdbeMemMakeWriteable(Mem * pMem)
-{
-	if ((pMem->flags & (MEM_Str | MEM_Blob)) != 0) {
-		if (ExpandBlob(pMem))
-			return -1;
-		if (pMem->szMalloc == 0 || pMem->z != pMem->zMalloc) {
-			if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
-				return -1;
-			}
-			pMem->z[pMem->n] = 0;
-			pMem->z[pMem->n + 1] = 0;
-			pMem->flags |= MEM_Term;
-		}
-	}
-	pMem->flags &= ~MEM_Ephem;
-#ifdef SQL_DEBUG
-	pMem->pScopyFrom = 0;
-#endif
-
-	return 0;
-}
-
 int
 sql_vdbemem_finalize(struct Mem *mem, struct func *func)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 58220d215..801f3cba6 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -179,6 +179,12 @@ mem_copy(struct Mem *to, const struct Mem *from);
 int
 mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
 
+/**
+ * Move all content of source MEM to destination MEM. Source MEM is set to NULL.
+ */
+int
+mem_move(struct Mem *to, struct Mem *from);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -495,8 +501,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
 
 int
 vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-void sqlVdbeMemMove(Mem *, Mem *);
-int sqlVdbeMemMakeWriteable(Mem *);
 
 /**
  * Memory cell mem contains the context of an aggregate function.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 55083fb23..4654b1610 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -165,21 +165,6 @@ vdbeTakeBranch(int iSrcLine, u8 I, u8 M)
 }
 #endif
 
-/*
- * An ephemeral string value (signified by the MEM_Ephem flag) contains
- * a pointer to a dynamically allocated string where some other entity
- * is responsible for deallocating that string.  Because the register
- * does not control the string, it might be deleted without the register
- * knowing it.
- *
- * This routine converts an ephemeral string into a dynamically allocated
- * string that the register itself controls.  In other words, it
- * converts an MEM_Ephem string into a string with P.z==P.zMalloc.
- */
-#define Deephemeralize(P)					\
-	if (((P)->flags&MEM_Ephem)!=0				\
-	    && sqlVdbeMemMakeWriteable(P)) { goto no_mem;}
-
 /* Return true if the cursor was opened using the OP_OpenSorter opcode. */
 #define isSorter(x) ((x)->eCurType==CURTYPE_SORTER)
 
@@ -949,13 +934,7 @@ case OP_Move: {
 		assert(pIn1<=&aMem[(p->nMem+1 - p->nCursor)]);
 		assert(memIsValid(pIn1));
 		memAboutToChange(p, pOut);
-		sqlVdbeMemMove(pOut, pIn1);
-#ifdef SQL_DEBUG
-		if (pOut->pScopyFrom>=&aMem[p1] && pOut->pScopyFrom<pOut) {
-			pOut->pScopyFrom += pOp->p2 - p1;
-		}
-#endif
-		Deephemeralize(pOut);
+		mem_move(pOut, pIn1);
 		REGISTER_TRACE(p, p2++, pOut);
 		pIn1++;
 		pOut++;
@@ -979,11 +958,7 @@ case OP_Copy: {
 	pOut = &aMem[pOp->p2];
 	assert(pOut!=pIn1);
 	while( 1) {
-		mem_copy_as_ephemeral(pOut, pIn1);
-		Deephemeralize(pOut);
-#ifdef SQL_DEBUG
-		pOut->pScopyFrom = 0;
-#endif
+		mem_copy(pOut, pIn1);
 		REGISTER_TRACE(p, pOp->p2+pOp->p3-n, pOut);
 		if ((n--)==0) break;
 		pOut++;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index b13385647..2625a9115 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -980,7 +980,7 @@ sqlTransferBindings(sql_stmt * pFromStmt, sql_stmt * pToStmt)
 	assert(pTo->db == pFrom->db);
 	assert(pTo->nVar == pFrom->nVar);
 	for (i = 0; i < pFrom->nVar; i++) {
-		sqlVdbeMemMove(&pTo->aVar[i], &pFrom->aVar[i]);
+		mem_move(&pTo->aVar[i], &pFrom->aVar[i]);
 	}
 	return 0;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 15/53] sql: rework vdbe_decode_msgpack_into_mem()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (13 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 14/53] sql: rework mem_move() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:02   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 16/53] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches
                   ` (37 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

The original vdbe_decode_msgpack_into_mem() returns a MEM that contains
string and binary values as ephemeral. This patch renames this function
to vdbe_decode_msgpack_into_mem_ephemeral() and introduces new
vdbe_decode_msgpack_into_mem(), which returns a MEM that contains string
and binary values in newly allocated memory.

This patch actually changes behavior in this case:

CREATE TABLE t1(m VARBINARY primary key);
INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379');
SELECT count(*), substr(m,2,1) AS m FROM t1 GROUP BY m;
SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;

But it doesn't change behaviour for this:

CREATE TABLE t2(m STRING primary key);
INSERT INTO t2 VALUES('ax'), ('bx'), ('cy');
SELECT count(*), substr(m,2,1) AS m FROM t2 GROUP BY m;
SELECT count(*), substr(m,2,1) AS mx FROM t2 GROUP BY mx;

Part of #5818
Part of #5890
---
 src/box/sql/mem.c     | 16 +++++++++++++++-
 src/box/sql/mem.h     | 17 ++++++++++++++++-
 src/box/sql/vdbe.c    | 18 ------------------
 src/box/sql/vdbeaux.c |  2 +-
 4 files changed, 32 insertions(+), 21 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 3d42ac63c..a2316cc90 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -2253,7 +2253,8 @@ sqlVdbeRecordCompareMsgpack(const void *key1,
 }
 
 int
-vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
+vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
+				       uint32_t *len)
 {
 	const char *start_buf = buf;
 	switch (mp_typeof(*buf)) {
@@ -2354,6 +2355,19 @@ install_blob:
 	return 0;
 }
 
+int
+vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
+{
+	if (vdbe_decode_msgpack_into_ephemeral_mem(buf, mem, len) != 0)
+		return -1;
+	if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) {
+		assert((mem->flags & MEM_Ephem) != 0);
+		if (sqlVdbeMemGrow(mem, mem->n, 1) != 0)
+			return -1;
+	}
+	return 0;
+}
+
 void
 mpstream_encode_vdbe_mem(struct mpstream *stream, struct Mem *var)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 801f3cba6..5edb0d80a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -538,7 +538,22 @@ int sqlVdbeRecordCompareMsgpack(const void *key1,
 				    struct UnpackedRecord *key2);
 
 /**
- * Decode msgpack and save value into VDBE memory cell.
+ * Decode msgpack and save value into VDBE memory cell. String and binary string
+ * values set as ephemeral.
+ *
+ * @param buf Buffer to deserialize msgpack from.
+ * @param mem Memory cell to write value into.
+ * @param len[out] Length of decoded part.
+ * @retval Return code: < 0 in case of error.
+ * @retval 0 on success.
+ */
+int
+vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
+				       uint32_t *len);
+
+/**
+ * Decode msgpack and save value into VDBE memory cell. String and binary string
+ * values copied to newly allocated memory.
  *
  * @param buf Buffer to deserialize msgpack from.
  * @param mem Memory cell to write value into.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4654b1610..d76ba0b40 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -375,24 +375,6 @@ vdbe_field_ref_fetch(struct vdbe_field_ref *field_ref, uint32_t fieldno,
 	uint32_t dummy;
 	if (vdbe_decode_msgpack_into_mem(data, dest_mem, &dummy) != 0)
 		return -1;
-
-	/*
-	 * Add 0 termination (at most for strings)
-	 * Not sure why do we check MEM_Ephem
-	 */
-	if (mem_is_string(dest_mem) && mem_is_ephemeral(dest_mem)) {
-		int len = dest_mem->n;
-		if (dest_mem->szMalloc < len + 1) {
-			if (sqlVdbeMemGrow(dest_mem, len + 1, 1) != 0)
-				return -1;
-		} else {
-			dest_mem->z =
-				memcpy(dest_mem->zMalloc, dest_mem->z, len);
-			dest_mem->flags &= ~MEM_Ephem;
-		}
-		dest_mem->z[len] = 0;
-		dest_mem->flags |= MEM_Term;
-	}
 	UPDATE_MAX_BLOBSIZE(dest_mem);
 	return 0;
 }
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 21ac84099..e58526401 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2334,7 +2334,7 @@ sqlVdbeRecordUnpackMsgpack(struct key_def *key_def,	/* Information about the rec
 		pMem->szMalloc = 0;
 		pMem->z = 0;
 		uint32_t sz = 0;
-		vdbe_decode_msgpack_into_mem(zParse, pMem, &sz);
+		vdbe_decode_msgpack_into_ephemeral_mem(zParse, pMem, &sz);
 		assert(sz != 0);
 		zParse += sz;
 		pMem++;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 16/53] sql: remove sql_column_to_messagepack()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (14 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 15/53] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 17/53] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches
                   ` (36 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

Function sql_column_to_messagepack() has almost the same functionality
as sql_vdbe_mem_encode_tuple(). Due to this it is not necessary to
have sql_column_to_messagepack(), so it is removed in this commit.

Part of #5818
---
 src/box/execute.c     | 130 ++----------------------------------------
 src/box/sql/mem.c     |   6 --
 src/box/sql/mem.h     |   3 -
 src/box/sql/sqlInt.h  |  15 +----
 src/box/sql/vdbeapi.c |  15 ++---
 5 files changed, 13 insertions(+), 156 deletions(-)

diff --git a/src/box/execute.c b/src/box/execute.c
index e14da20be..f9c161ce4 100644
--- a/src/box/execute.c
+++ b/src/box/execute.c
@@ -131,113 +131,10 @@ port_sql_create(struct port *port, struct sql_stmt *stmt,
 	port_sql->do_finalize = do_finalize;
 }
 
-/**
- * Serialize a single column of a result set row.
- * @param stmt Prepared and started statement. At least one
- *        sql_step must be called.
- * @param i Column number.
- * @param region Allocator for column value.
- *
- * @retval  0 Success.
- * @retval -1 Out of memory when resizing the output buffer.
- */
-static inline int
-sql_column_to_messagepack(struct sql_stmt *stmt, int i,
-			  struct region *region)
-{
-	size_t size;
-	enum mp_type type = sql_column_type(stmt, i);
-	switch (type) {
-	case MP_INT: {
-		int64_t n = sql_column_int64(stmt, i);
-		size = mp_sizeof_int(n);
-		char *pos = (char *) region_alloc(region, size);
-		if (pos == NULL)
-			goto oom;
-		mp_encode_int(pos, n);
-		break;
-	}
-	case MP_UINT: {
-		uint64_t n = sql_column_uint64(stmt, i);
-		size = mp_sizeof_uint(n);
-		char *pos = (char *) region_alloc(region, size);
-		if (pos == NULL)
-			goto oom;
-		mp_encode_uint(pos, n);
-		break;
-	}
-	case MP_DOUBLE: {
-		double d = sql_column_double(stmt, i);
-		size = mp_sizeof_double(d);
-		char *pos = (char *) region_alloc(region, size);
-		if (pos == NULL)
-			goto oom;
-		mp_encode_double(pos, d);
-		break;
-	}
-	case MP_STR: {
-		uint32_t len = sql_column_bytes(stmt, i);
-		size = mp_sizeof_str(len);
-		char *pos = (char *) region_alloc(region, size);
-		if (pos == NULL)
-			goto oom;
-		const char *s;
-		s = (const char *)sql_column_text(stmt, i);
-		mp_encode_str(pos, s, len);
-		break;
-	}
-	case MP_BIN:
-	case MP_MAP:
-	case MP_ARRAY: {
-		uint32_t len = sql_column_bytes(stmt, i);
-		const char *s =
-			(const char *)sql_column_blob(stmt, i);
-		if (sql_column_subtype(stmt, i) == SQL_SUBTYPE_MSGPACK) {
-			size = len;
-			char *pos = (char *)region_alloc(region, size);
-			if (pos == NULL)
-				goto oom;
-			memcpy(pos, s, len);
-		} else {
-			size = mp_sizeof_bin(len);
-			char *pos = (char *)region_alloc(region, size);
-			if (pos == NULL)
-				goto oom;
-			mp_encode_bin(pos, s, len);
-		}
-		break;
-	}
-	case MP_BOOL: {
-		bool b = sql_column_boolean(stmt, i);
-		size = mp_sizeof_bool(b);
-		char *pos = (char *) region_alloc(region, size);
-		if (pos == NULL)
-			goto oom;
-		mp_encode_bool(pos, b);
-		break;
-	}
-	case MP_NIL: {
-		size = mp_sizeof_nil();
-		char *pos = (char *) region_alloc(region, size);
-		if (pos == NULL)
-			goto oom;
-		mp_encode_nil(pos);
-		break;
-	}
-	default:
-		unreachable();
-	}
-	return 0;
-oom:
-	diag_set(OutOfMemory, size, "region_alloc", "SQL value");
-	return -1;
-}
-
 /**
  * Convert sql row into a tuple and append to a port.
  * @param stmt Started prepared statement. At least one
  *        sql_step must be done.
- * @param column_count Statement's column count.
  * @param region Runtime allocator for temporary objects.
  * @param port Port to store tuples.
  *
@@ -245,29 +142,11 @@ oom:
  * @retval -1 Memory error.
  */
 static inline int
-sql_row_to_port(struct sql_stmt *stmt, int column_count,
-		struct region *region, struct port *port)
+sql_row_to_port(struct sql_stmt *stmt, struct region *region, struct port *port)
 {
-	assert(column_count > 0);
-	size_t size = mp_sizeof_array(column_count);
+	uint32_t size;
 	size_t svp = region_used(region);
-	char *pos = (char *) region_alloc(region, size);
-	if (pos == NULL) {
-		diag_set(OutOfMemory, size, "region_alloc", "SQL row");
-		return -1;
-	}
-	mp_encode_array(pos, column_count);
-
-	for (int i = 0; i < column_count; ++i) {
-		if (sql_column_to_messagepack(stmt, i, region) != 0)
-			goto error;
-	}
-	size = region_used(region) - svp;
-	pos = (char *) region_join(region, size);
-	if (pos == NULL) {
-		diag_set(OutOfMemory, size, "region_join", "pos");
-		goto error;
-	}
+	char *pos = sql_result_to_msgpack(stmt, &size, region);
 	struct tuple *tuple =
 		tuple_new(box_tuple_format_default(), pos, pos + size);
 	if (tuple == NULL)
@@ -677,8 +556,7 @@ sql_execute(struct sql_stmt *stmt, struct port *port, struct region *region)
 	if (column_count > 0) {
 		/* Either ROW or DONE or ERROR. */
 		while ((rc = sql_step(stmt)) == SQL_ROW) {
-			if (sql_row_to_port(stmt, column_count, region,
-					    port) != 0)
+			if (sql_row_to_port(stmt, region, port) != 0)
 				return -1;
 		}
 		assert(rc == SQL_DONE || rc != 0);
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index a2316cc90..775474f69 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1791,12 +1791,6 @@ sqlValueText(sql_value * pVal)
 	return valueToText(pVal);
 }
 
-enum sql_subtype
-sql_value_subtype(sql_value * pVal)
-{
-	return (pVal->flags & MEM_Subtype) != 0 ? pVal->subtype : SQL_SUBTYPE_NO;
-}
-
 /*
  * Return a pointer to static memory containing an SQL NULL value.
  */
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 5edb0d80a..aead71df2 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -467,9 +467,6 @@ const void *sqlValueText(struct Mem *);
 
 #define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
 
-enum sql_subtype
-sql_value_subtype(sql_value * pVal);
-
 const Mem *
 columnNullValue(void);
 
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index b4293d961..01c0cc5a1 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -351,16 +351,6 @@ sql_stricmp(const char *, const char *);
 int
 sql_strnicmp(const char *, const char *, int);
 
-
-/**
- * Get row column subtype.
- * @param stmt row data to process.
- * @param i column index.
- * @retval SQL subtype if any, 0 else.
- */
-enum sql_subtype
-sql_column_subtype(struct sql_stmt *stmt, int i);
-
 sql *
 sql_context_db_handle(sql_context *);
 
@@ -471,8 +461,9 @@ const unsigned char *
 sql_column_text(sql_stmt *,
 		    int iCol);
 
-enum mp_type
-sql_column_type(sql_stmt *stmt, int field_no);
+char *
+sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
+		      struct region *region);
 
 /*
  * Terminate the current execution of an SQL statement and reset
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 2625a9115..04016ea54 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -515,16 +515,13 @@ sql_column_text(sql_stmt * pStmt, int i)
 	return sql_value_text(columnMem(pStmt, i));
 }
 
-enum mp_type
-sql_column_type(sql_stmt * pStmt, int i)
-{
-	return sql_value_type(columnMem(pStmt, i));
-}
-
-enum sql_subtype
-sql_column_subtype(struct sql_stmt *stmt, int i)
+char *
+sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
+		      struct region *region)
 {
-	return sql_value_subtype(columnMem(stmt, i));
+	struct Vdbe *vdbe = (struct Vdbe *)stmt;
+	return sql_vdbe_mem_encode_tuple(vdbe->pResultSet, vdbe->nResColumn,
+					 tuple_size, region);
 }
 
 /*
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 17/53] sql: introduce mem_concat()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (15 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 16/53] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 18/53] sql: introduce mem_arithmetic() Mergen Imeev via Tarantool-patches
                   ` (35 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_concat(). Function mem_concat() concatenates
values from two MEMs in case these values are strings or binaries and
writes the result to the third MEM.

Part of #5818
---
 src/box/sql/mem.c  | 51 +++++++++++++++++++++++++++++++++++++++++++++
 src/box/sql/mem.h  |  7 +++++++
 src/box/sql/vdbe.c | 52 ++--------------------------------------------
 3 files changed, 60 insertions(+), 50 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 775474f69..f160439c9 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -319,6 +319,57 @@ mem_move(struct Mem *to, struct Mem *from)
 	return 0;
 }
 
+int
+mem_concat(struct Mem *left, struct Mem *right, struct Mem *result)
+{
+	assert(result != right);
+	sqlVdbeMemSetNull(result);
+	result->field_type = field_type_MAX;
+
+	if (((left->flags | right->flags) & MEM_Null) != 0) {
+		/* Force NULL be of type STRING. */
+		result->field_type = FIELD_TYPE_STRING;
+		return 0;
+	}
+
+	/* Concatenation operation can be applied only to strings and blobs. */
+	if ((right->flags & (MEM_Str | MEM_Blob)) == 0) {
+		diag_set(ClientError, ER_INCONSISTENT_TYPES,
+			 "text or varbinary", mem_type_to_str(right));
+		return -1;
+	}
+	if ((left->flags & (MEM_Str | MEM_Blob)) == 0) {
+		diag_set(ClientError, ER_INCONSISTENT_TYPES,
+			 "text or varbinary", mem_type_to_str(left));
+		return -1;
+	}
+
+	/* Moreover, both operands must be of the same type. */
+	if ((right->flags & MEM_Str) != (left->flags & MEM_Str)) {
+		diag_set(ClientError, ER_INCONSISTENT_TYPES,
+			 mem_type_to_str(left), mem_type_to_str(right));
+		return -1;
+	}
+
+	if (ExpandBlob(left) != 0 || ExpandBlob(right) != 0)
+		return -1;
+
+	uint32_t size = left->n + right->n;
+	if ((int)size > sql_get()->aLimit[SQL_LIMIT_LENGTH]) {
+		diag_set(ClientError, ER_SQL_EXECUTE, "string or blob too big");
+		return -1;
+	}
+	if (sqlVdbeMemGrow(result, size, result == left) != 0)
+		return -1;
+
+	result->flags = left->flags & (MEM_Str | MEM_Blob);
+	if (result != left)
+		memcpy(result->z, left->z, left->n);
+	memcpy(&result->z[left->n], right->z, right->n);
+	result->n = size;
+	return 0;
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index aead71df2..9b0eb8444 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -185,6 +185,13 @@ mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from);
 int
 mem_move(struct Mem *to, struct Mem *from);
 
+/**
+ * Concatenate strings or binaries from the first and the second MEMs and write
+ * the result to the third MEM.
+ */
+int
+mem_concat(struct Mem *left, struct Mem *right, struct Mem *result);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index d76ba0b40..c04bd40f3 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1028,59 +1028,11 @@ case OP_ResultRow: {
  * types (i.e. TEXT and BLOB).
  */
 case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
-	i64 nByte;
-
 	pIn1 = &aMem[pOp->p1];
 	pIn2 = &aMem[pOp->p2];
-	pOut = vdbe_prepare_null_out(p, pOp->p3);
-	assert(pIn1!=pOut);
-	if (mem_is_null(pIn1) || mem_is_null(pIn2)) {
-		/* Force NULL be of type STRING. */
-		pOut->field_type = FIELD_TYPE_STRING;
-		break;
-	}
-	/*
-	 * Concatenation operation can be applied only to
-	 * strings and blobs.
-	 */
-	bool str_type_p1 = mem_is_varstring(pIn1);
-	bool str_type_p2 = mem_is_varstring(pIn2);
-	if (!str_type_p1 || !str_type_p2) {
-		char *inconsistent_type = !str_type_p1 ?
-					  mem_type_to_str(pIn1) :
-					  mem_type_to_str(pIn2);
-		diag_set(ClientError, ER_INCONSISTENT_TYPES,
-			 "text or varbinary", inconsistent_type);
-		goto abort_due_to_error;
-	}
-
-	/* Moreover, both operands must be of the same type. */
-	if (mem_is_string(pIn1) != mem_is_string(pIn2)) {
-		diag_set(ClientError, ER_INCONSISTENT_TYPES,
-			 mem_type_to_str(pIn2), mem_type_to_str(pIn1));
-		goto abort_due_to_error;
-	}
-	if (ExpandBlob(pIn1) != 0 || ExpandBlob(pIn2) != 0)
+	pOut = &aMem[pOp->p3];
+	if (mem_concat(pIn2, pIn1, pOut) != 0)
 		goto abort_due_to_error;
-	nByte = pIn1->n + pIn2->n;
-	if (nByte>db->aLimit[SQL_LIMIT_LENGTH]) {
-		goto too_big;
-	}
-	if (sqlVdbeMemGrow(pOut, (int)nByte+2, pOut==pIn2)) {
-		goto no_mem;
-	}
-	if (mem_is_string(pIn1))
-		MemSetTypeFlag(pOut, MEM_Str);
-	else
-		MemSetTypeFlag(pOut, MEM_Blob);
-	if (pOut!=pIn2) {
-		memcpy(pOut->z, pIn2->z, pIn2->n);
-	}
-	memcpy(&pOut->z[pIn2->n], pIn1->z, pIn1->n);
-	pOut->z[nByte]=0;
-	pOut->z[nByte+1] = 0;
-	pOut->flags |= MEM_Term;
-	pOut->n = (int)nByte;
 	UPDATE_MAX_BLOBSIZE(pOut);
 	break;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 18/53] sql: introduce mem_arithmetic()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (16 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 17/53] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:02   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 19/53] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
                   ` (34 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_arithmetic(). Function mem_arithmetic()
executes arithmetic operations on the first and the second MEMs and
writes the result to the third MEM.

Part of #5818
---
 src/box/sql/mem.c  | 224 +++++++++++++++++++++++++++++++++++++--------
 src/box/sql/mem.h  |   6 +-
 src/box/sql/vdbe.c | 111 +---------------------
 3 files changed, 193 insertions(+), 148 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index f160439c9..6120939d8 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -370,6 +370,192 @@ mem_concat(struct Mem *left, struct Mem *right, struct Mem *result)
 	return 0;
 }
 
+int
+mem_arithmetic(const struct Mem *left, const struct Mem *right,
+	       struct Mem *result, int op)
+{
+	sqlVdbeMemSetNull(result);
+	result->field_type = FIELD_TYPE_NUMBER;
+	if (((left->flags | right->flags) & MEM_Null) != 0)
+		return 0;
+
+	int64_t il;
+	bool is_l_neg;
+	double dl;
+	uint16_t type_left = 0;
+	if ((left->flags & MEM_Real) != 0) {
+		dl = left->u.r;
+		type_left = MEM_Real;
+	} else if ((left->flags & MEM_Int) != 0) {
+		il = left->u.i;
+		type_left = MEM_Int;
+		is_l_neg = true;
+	} else if ((left->flags & MEM_UInt) != 0) {
+		il = left->u.i;
+		type_left = MEM_UInt;
+		is_l_neg = false;
+	} else if ((left->flags & (MEM_Str | MEM_Blob)) != 0) {
+		if (sql_atoi64(left->z, &il, &is_l_neg, left->n) == 0)
+			type_left = is_l_neg ? MEM_Int : MEM_UInt;
+		else if (sqlAtoF(left->z, &dl, left->n) != 0)
+			type_left = MEM_Real;
+	}
+
+	int64_t ir;
+	bool is_r_neg;
+	double dr;
+	uint16_t type_right = 0;
+	if ((right->flags & MEM_Real) != 0) {
+		dr = right->u.r;
+		type_right = MEM_Real;
+	} else if ((right->flags & MEM_Int) != 0) {
+		ir = right->u.i;
+		type_right = MEM_Int;
+		is_r_neg = true;
+	} else if ((right->flags & MEM_UInt) != 0) {
+		ir = right->u.i;
+		type_right = MEM_UInt;
+		is_r_neg = false;
+	} else if ((right->flags & (MEM_Str | MEM_Blob)) != 0) {
+		if (sql_atoi64(right->z, &ir, &is_r_neg, right->n) == 0)
+			type_right = is_r_neg ? MEM_Int : MEM_UInt;
+		else if (sqlAtoF(right->z, &dr, right->n) != 0)
+			type_right = MEM_Real;
+	}
+
+	if ((type_right & (MEM_Int | MEM_UInt | MEM_Real)) == 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(right), "numeric");
+		return -1;
+	}
+	if ((type_left & (MEM_Int | MEM_UInt | MEM_Real)) == 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(left), "numeric");
+		return -1;
+	}
+	if (((type_left | type_right) & MEM_Real) != 0) {
+		if (type_left == MEM_Int)
+			dl = (double)il;
+		else if (type_left == MEM_UInt)
+			dl = (double)(uint64_t)il;
+
+		if (type_right == MEM_Int)
+			dr = (double)ir;
+		else if (type_right == MEM_UInt)
+			dr = (double)(uint64_t)ir;
+
+		double dres;
+		switch(op) {
+		case OP_Add:
+			dres = dl + dr;
+			break;
+		case OP_Subtract:
+			dres = dl - dr;
+			break;
+		case OP_Multiply:
+			dres = dl * dr;
+			break;
+		case OP_Divide:
+			if (dr == 0.) {
+				diag_set(ClientError, ER_SQL_EXECUTE,
+					 "division by zero");
+				return -1;
+			}
+			dres = dl / dr;
+			break;
+		case OP_Remainder: {
+			int64_t il = (int64_t)dl;
+			int64_t ir = (int64_t)dr;
+			if (ir == 0) {
+				diag_set(ClientError, ER_SQL_EXECUTE,
+					 "division by zero");
+				return -1;
+			}
+			if (ir == -1)
+				ir = 1;
+			dres = (double)(il % ir);
+			break;
+		}
+		default:
+			unreachable();
+		}
+		if (sqlIsNaN(dres))
+			return 0;
+		result->u.r = dres;
+		result->flags = MEM_Real;
+		return 0;
+	}
+	int64_t ires;
+	/*
+	 * TODO: This is wrong. Both these flags should already be set. This
+	 * assignment done to not change behaviour of the function, which
+	 * is currently bugged.
+	 */
+	is_l_neg = (left->flags & MEM_Int) != 0;
+	is_r_neg = (right->flags & MEM_Int) != 0;
+	bool is_res_neg;
+	switch(op) {
+	case OP_Add:
+		if (sql_add_int(il, is_l_neg, ir, is_r_neg, &ires,
+				&is_res_neg) != 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "integer is overflowed");
+			return -1;
+		}
+		break;
+	case OP_Subtract:
+		if (sql_sub_int(il, is_l_neg, ir, is_r_neg, &ires,
+				&is_res_neg) != 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "integer is overflowed");
+			return -1;
+		}
+		break;
+	case OP_Multiply:
+		if (sql_mul_int(il, is_l_neg, ir, is_r_neg, &ires,
+				&is_res_neg) != 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "integer is overflowed");
+			return -1;
+		}
+		break;
+	case OP_Divide:
+		if (ir == 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "division by zero");
+			return -1;
+		}
+		if (sql_div_int(il, is_l_neg, ir, is_r_neg, &ires,
+				&is_res_neg) != 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "integer is overflowed");
+			return -1;
+		}
+		break;
+	case OP_Remainder: {
+		if (ir == 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "division by zero");
+			return -1;
+		}
+		if (ir == -1)
+			ir = 1;
+		if (sql_rem_int(il, is_l_neg, ir, is_r_neg, &ires,
+				&is_res_neg) != 0) {
+			diag_set(ClientError, ER_SQL_EXECUTE,
+				 "integer is overflowed");
+			return -1;
+		}
+		break;
+	}
+	default:
+		unreachable();
+	}
+	result->u.i = ires;
+	result->flags = is_res_neg ? MEM_Int : MEM_UInt;
+	return 0;
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
@@ -574,44 +760,6 @@ sql_value_type(sql_value *pVal)
 	return mem_mp_type(mem);
 }
 
-
-/*
- * pMem currently only holds a string type (or maybe a BLOB that we can
- * interpret as a string if we want to).  Compute its corresponding
- * numeric type, if has one.  Set the pMem->u.r and pMem->u.i fields
- * accordingly.
- */
-static u16 SQL_NOINLINE
-computeNumericType(Mem *pMem)
-{
-	assert((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) == 0);
-	assert((pMem->flags & (MEM_Str|MEM_Blob))!=0);
-	if (sqlAtoF(pMem->z, &pMem->u.r, pMem->n)==0)
-		return 0;
-	bool is_neg;
-	if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, &is_neg, pMem->n) == 0)
-		return is_neg ? MEM_Int : MEM_UInt;
-	return MEM_Real;
-}
-
-/*
- * Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
- * none.
- *
- * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
- * But it does set pMem->u.r and pMem->u.i appropriately.
- */
-u16
-numericType(Mem *pMem)
-{
-	if ((pMem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
-		return pMem->flags & (MEM_Int | MEM_UInt | MEM_Real);
-	if (pMem->flags & (MEM_Str|MEM_Blob)) {
-		return computeNumericType(pMem);
-	}
-	return 0;
-}
-
 /*
  * The sqlValueBytes() routine returns the number of bytes in the
  * sql_value object assuming that it uses the encoding "enc".
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9b0eb8444..7cea07dc9 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -192,6 +192,10 @@ mem_move(struct Mem *to, struct Mem *from);
 int
 mem_concat(struct Mem *left, struct Mem *right, struct Mem *result);
 
+int
+mem_arithmetic(const struct Mem *left, const struct Mem *right,
+	       struct Mem *result, int op);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
@@ -261,8 +265,6 @@ mem_mp_type(struct Mem *mem);
 
 enum mp_type
 sql_value_type(struct Mem *);
-u16
-numericType(Mem *pMem);
 
 int sqlValueBytes(struct Mem *);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index c04bd40f3..55cb42932 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1080,117 +1080,12 @@ case OP_Subtract:              /* same as TK_MINUS, in1, in2, out3 */
 case OP_Multiply:              /* same as TK_STAR, in1, in2, out3 */
 case OP_Divide:                /* same as TK_SLASH, in1, in2, out3 */
 case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
-	u16 type1;      /* Numeric type of left operand */
-	u16 type2;      /* Numeric type of right operand */
-	i64 iA;         /* Integer value of left operand */
-	i64 iB;         /* Integer value of right operand */
-	double rA;      /* Real value of left operand */
-	double rB;      /* Real value of right operand */
-
 	pIn1 = &aMem[pOp->p1];
-	type1 = numericType(pIn1);
 	pIn2 = &aMem[pOp->p2];
-	type2 = numericType(pIn2);
-	pOut = vdbe_prepare_null_out(p, pOp->p3);
-	if (mem_is_null(pIn1) || mem_is_null(pIn2))
-		goto arithmetic_result_is_null;
-	if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
-	    (type2 & (MEM_Int | MEM_UInt)) != 0) {
-		iA = pIn1->u.i;
-		iB = pIn2->u.i;
-		bool is_lhs_neg = mem_is_integer(pIn1) &&
-				  !mem_is_unsigned(pIn1);
-		bool is_rhs_neg = mem_is_integer(pIn2) &&
-				  !mem_is_unsigned(pIn2);
-		bool is_res_neg;
-		switch( pOp->opcode) {
-		case OP_Add: {
-			if (sql_add_int(iA, is_lhs_neg, iB, is_rhs_neg,
-					(int64_t *) &iB, &is_res_neg) != 0)
-				goto integer_overflow;
-			break;
-		}
-		case OP_Subtract: {
-			if (sql_sub_int(iB, is_rhs_neg, iA, is_lhs_neg,
-					(int64_t *) &iB, &is_res_neg) != 0)
-				goto integer_overflow;
-			break;
-		}
-		case OP_Multiply: {
-			if (sql_mul_int(iA, is_lhs_neg, iB, is_rhs_neg,
-					(int64_t *) &iB, &is_res_neg) != 0)
-				goto integer_overflow;
-			break;
-		}
-		case OP_Divide: {
-			if (iA == 0)
-				goto division_by_zero;
-			if (sql_div_int(iB, is_rhs_neg, iA, is_lhs_neg,
-					(int64_t *) &iB, &is_res_neg) != 0)
-				goto integer_overflow;
-			break;
-		}
-		default: {
-			if (iA == 0)
-				goto division_by_zero;
-			if (iA==-1) iA = 1;
-			if (sql_rem_int(iB, is_rhs_neg, iA, is_lhs_neg,
-					(int64_t *) &iB, &is_res_neg) != 0)
-				goto integer_overflow;
-			break;
-		}
-		}
-		mem_set_int(pOut, iB, is_res_neg);
-	} else {
-		if (sqlVdbeRealValue(pIn1, &rA) != 0) {
-			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 mem_str(pIn1), "numeric");
-			goto abort_due_to_error;
-		}
-		if (sqlVdbeRealValue(pIn2, &rB) != 0) {
-			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 mem_str(pIn2), "numeric");
-			goto abort_due_to_error;
-		}
-		assert(((type1 | type2) & MEM_Real) != 0);
-		switch( pOp->opcode) {
-		case OP_Add:         rB += rA;       break;
-		case OP_Subtract:    rB -= rA;       break;
-		case OP_Multiply:    rB *= rA;       break;
-		case OP_Divide: {
-			if (rA == (double)0)
-				goto division_by_zero;
-			rB /= rA;
-			break;
-		}
-		default: {
-			iA = (i64)rA;
-			iB = (i64)rB;
-			if (iA == 0)
-				goto division_by_zero;
-			if (iA==-1) iA = 1;
-			rB = (double)(iB % iA);
-			break;
-		}
-		}
-		if (sqlIsNaN(rB)) {
-			goto arithmetic_result_is_null;
-		}
-		mem_set_double(pOut, rB);
-	}
-	break;
-
-arithmetic_result_is_null:
-	/* Force NULL be of type NUMBER. */
-	pOut->field_type = FIELD_TYPE_NUMBER;
+	pOut = &aMem[pOp->p3];
+	if (mem_arithmetic(pIn2, pIn1, pOut, pOp->opcode) != 0)
+		goto abort_due_to_error;
 	break;
-
-division_by_zero:
-	diag_set(ClientError, ER_SQL_EXECUTE, "division by zero");
-	goto abort_due_to_error;
-integer_overflow:
-	diag_set(ClientError, ER_SQL_EXECUTE, "integer is overflowed");
-	goto abort_due_to_error;
 }
 
 /* Opcode: CollSeq P1 * * P4
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 19/53] sql: introduce mem_compare()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (17 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 18/53] sql: introduce mem_arithmetic() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:03   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 20/53] sql: introduce mem_bitwise() Mergen Imeev via Tarantool-patches
                   ` (33 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_compare(). Function mem_compare() compares the
first and the second MEMs and returns result of comparison.

Part of #5818
---
 src/box/sql/mem.c  | 241 +++++++++++++++++++++++++++++++++++++++++++++
 src/box/sql/mem.h  |   4 +
 src/box/sql/vdbe.c |  92 +----------------
 3 files changed, 247 insertions(+), 90 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 6120939d8..8119644ed 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -556,6 +556,247 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
 	return 0;
 }
 
+static int
+compare_blobs(const struct Mem *left, const struct Mem *right, int *result)
+{
+	int nl = left->n;
+	int nr = right->n;
+	int minlen = MIN(nl, nr);
+
+	/*
+	 * It is possible to have a Blob value that has some non-zero content
+	 * followed by zero content.  But that only comes up for Blobs formed
+	 * by the OP_MakeRecord opcode, and such Blobs never get passed into
+	 * mem_compare().
+	 */
+	assert((left->flags & MEM_Zero) == 0 || nl == 0);
+	assert((right->flags & MEM_Zero) == 0 || nr == 0);
+
+	if (left->flags & right->flags & MEM_Zero) {
+		*result = left->u.nZero - right->u.nZero;
+		return 0;
+	}
+	if (left->flags & MEM_Zero) {
+		for (int i = 0; i < minlen; ++i) {
+			if (right->z[i] != 0) {
+				*result = -1;
+				return 0;
+			}
+		}
+		*result = left->u.nZero - nr;
+		return 0;
+	}
+	if (right->flags & MEM_Zero) {
+		for (int i = 0; i < minlen; ++i) {
+			if (left->z[i] != 0){
+				*result = 1;
+				return 0;
+			}
+		}
+		*result = right->u.nZero - nl;
+		return 0;
+	}
+	*result = memcmp(left->z, right->z, minlen);
+	if (*result != 0)
+		return 0;
+	*result = nl - nr;
+	return 0;
+}
+
+static int
+compare_numbers(const struct Mem *left, const struct Mem *right, int *result)
+{
+	int64_t il;
+	double dl;
+	int left_type = left->flags & (MEM_Int | MEM_UInt | MEM_Real);
+	if (left_type == MEM_Real) {
+		dl = left->u.r;
+	} else if (left_type != 0) {
+		il = left->u.i;
+	} else if ((left->flags & MEM_Str) != 0) {
+		bool is_l_neg;
+		if (sql_atoi64(left->z, &il, &is_l_neg, left->n) == 0)
+			left_type = is_l_neg ? MEM_Int : MEM_UInt;
+		else if (sqlAtoF(left->z, &dl, left->n) != 0)
+			left_type = MEM_Real;
+	}
+	if (left_type == 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(left), "numeric");
+		return -1;
+	}
+
+	int64_t ir;
+	double dr;
+	int right_type = right->flags & (MEM_Int | MEM_UInt | MEM_Real);
+	if (right_type == MEM_Real) {
+		dr = right->u.r;
+	} else if (right_type != 0) {
+		ir = right->u.i;
+	} else if ((right->flags & MEM_Str) != 0) {
+		bool is_r_neg;
+		if (sql_atoi64(right->z, &ir, &is_r_neg, right->n) == 0)
+			right_type = is_r_neg ? MEM_Int : MEM_UInt;
+		else if (sqlAtoF(right->z, &dr, right->n) != 0)
+			right_type = MEM_Real;
+	}
+	/* TODO: There should be check for rvalue type. */
+	if (right_type == 0) {
+		*result = -1;
+		return 0;
+	}
+	if (left_type == right_type) {
+		if (left_type == MEM_Real) {
+			if (dl > dr)
+				*result = 1;
+			else if (dl < dr)
+				*result = -1;
+			else
+				*result = 0;
+			return 0;
+		}
+		if (left_type == MEM_Int) {
+			if (il > ir)
+				*result = 1;
+			else if (il < ir)
+				*result = -1;
+			else
+				*result = 0;
+			return 0;
+		}
+		uint64_t ul = (uint64_t)il;
+		uint64_t ur = (uint64_t)ir;
+		if (ul > ur)
+			*result = 1;
+		else if (ul < ur)
+			*result = -1;
+		else
+			*result = 0;
+		return 0;
+	}
+	if (left_type == MEM_Real) {
+		if (right_type == MEM_Int) {
+			*result = double_compare_nint64(dl, ir, 1);
+			return 0;
+		}
+		*result = double_compare_uint64(dl, (uint64_t)ir, 1);
+		return 0;
+	}
+	if (right_type == MEM_Real) {
+		if (left_type == MEM_Int) {
+			*result = double_compare_nint64(dr, il, -1);
+			return 0;
+		}
+		*result = double_compare_uint64(dr, (uint64_t)il, -1);
+		return 0;
+	}
+	if (left_type == MEM_Int) {
+		*result = -1;
+		return 0;
+	}
+	assert(right_type == MEM_Int && left_type == MEM_UInt);
+	*result = 1;
+	return 0;
+}
+
+static int
+compare_strings(const struct Mem *left, const struct Mem *right, int *result,
+		struct coll *coll)
+{
+	char *sl;
+	uint32_t nl;
+	char bufl[BUF_SIZE];
+	if ((left->flags & MEM_Str) != 0) {
+		sl = left->z;
+		nl = left->n;
+	} else {
+		assert((left->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+		sl = &bufl[0];
+		if ((left->flags & MEM_Int) != 0)
+			sql_snprintf(BUF_SIZE, sl, "%lld", left->u.i);
+		else if ((left->flags & MEM_UInt) != 0)
+			sql_snprintf(BUF_SIZE, sl, "%llu", left->u.u);
+		else
+			sql_snprintf(BUF_SIZE, sl, "%!.15g", left->u.r);
+		nl = strlen(sl);
+	}
+
+	char *sr;
+	uint32_t nr;
+	char bufr[BUF_SIZE];
+	if ((right->flags & MEM_Str) != 0) {
+		sr = right->z;
+		nr = right->n;
+	} else {
+		assert((right->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0);
+		sr = &bufr[0];
+		if ((right->flags & MEM_Int) != 0)
+			sql_snprintf(BUF_SIZE, sr, "%lld", right->u.i);
+		else if ((right->flags & MEM_UInt) != 0)
+			sql_snprintf(BUF_SIZE, sr, "%llu", right->u.u);
+		else
+			sql_snprintf(BUF_SIZE, sr, "%!.15g", right->u.r);
+		nr = strlen(sr);
+	}
+	if (coll) {
+		*result = coll->cmp(sl, nl, sr, nr, coll);
+		return 0;
+	}
+	uint32_t minlen = MIN(nl, nr);
+	*result = memcmp(sl, sr, minlen);
+	if (*result != 0)
+		return 0;
+	*result = nl - nr;
+	return 0;
+}
+
+int
+mem_compare(const struct Mem *left, const struct Mem *right, int *result,
+	    enum field_type type, struct coll *coll)
+{
+	assert(((left->flags | right->flags) & MEM_Null) == 0);
+	if ((right->flags & MEM_Bool) != 0) {
+		if ((left->flags & MEM_Bool) != 0) {
+			if (left->u.b == right->u.b)
+				*result = 0;
+			else if (left->u.b)
+				*result = 1;
+			else
+				*result = -1;
+			return 0;
+		}
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_type_to_str(left), "boolean");
+		return -1;
+	}
+	if ((left->flags & MEM_Bool) != 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_type_to_str(right), "boolean");
+		return -1;
+	}
+	if ((right->flags & MEM_Blob) != 0) {
+		if ((left->flags & MEM_Blob) != 0)
+			return compare_blobs(left, right, result);
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_type_to_str(left), "varbinary");
+		return -1;
+	}
+	if ((left->flags & MEM_Blob) != 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_type_to_str(right), "varbinary");
+		return -1;
+	}
+	if (sql_type_is_numeric(type))
+		return compare_numbers(left, right, result);
+	if (type == FIELD_TYPE_STRING)
+		return compare_strings(left, right, result, coll);
+	if (((left->flags | right->flags) &
+	     (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+		return compare_numbers(left, right, result);
+	assert((left->flags & MEM_Str) != 0 && (right->flags & MEM_Str) != 0);
+	return compare_strings(left, right, result, coll);
+}
+
 static inline bool
 mem_has_msgpack_subtype(struct Mem *mem)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 7cea07dc9..096ce6533 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -196,6 +196,10 @@ int
 mem_arithmetic(const struct Mem *left, const struct Mem *right,
 	       struct Mem *result, int op);
 
+int
+mem_compare(const struct Mem *left, const struct Mem *right, int *result,
+	    enum field_type type, struct coll *coll);
+
 /* One or more of the following flags are set to indicate the validOK
  * representations of the value stored in the Mem struct.
  *
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 55cb42932..726066c17 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1543,15 +1543,9 @@ case OP_Le:               /* same as TK_LE, jump, in1, in3 */
 case OP_Gt:               /* same as TK_GT, jump, in1, in3 */
 case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 	int res, res2;      /* Result of the comparison of pIn1 against pIn3 */
-	u32 flags1;         /* Copy of initial value of pIn1->flags */
-	u32 flags3;         /* Copy of initial value of pIn3->flags */
 
 	pIn1 = &aMem[pOp->p1];
 	pIn3 = &aMem[pOp->p3];
-	flags1 = pIn1->flags;
-	flags3 = pIn3->flags;
-	enum field_type ft_p1 = pIn1->field_type;
-	enum field_type ft_p3 = pIn3->field_type;
 	if (mem_is_null(pIn1) || mem_is_null(pIn3)) {
 		/* One or both operands are NULL */
 		if (pOp->p5 & SQL_NULLEQ) {
@@ -1585,85 +1579,11 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 			}
 			break;
 		}
-	} else if (mem_is_boolean(pIn1) || mem_is_boolean(pIn3) ||
-		   mem_is_binary(pIn1) || mem_is_binary(pIn3)) {
-		if (!mem_is_same_type(pIn1, pIn3)) {
-			char *inconsistent_type = mem_is_boolean(pIn1) ||
-						  mem_is_binary(pIn1) ?
-						  mem_type_to_str(pIn3) :
-						  mem_type_to_str(pIn1);
-			char *expected_type = mem_is_boolean(pIn1) ||
-					      mem_is_binary(pIn1) ?
-					      mem_type_to_str(pIn1) :
-					      mem_type_to_str(pIn3);
-			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 inconsistent_type, expected_type);
-			goto abort_due_to_error;
-		}
-		res = sqlMemCompare(pIn3, pIn1, NULL);
 	} else {
 		enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
-		if (sql_type_is_numeric(type)) {
-			if (mem_is_string(pIn1)) {
-				mem_apply_numeric_type(pIn1);
-				flags3 = pIn3->flags;
-			}
-			if (mem_is_string(pIn3)) {
-				if (mem_apply_numeric_type(pIn3) != 0) {
-					diag_set(ClientError,
-						 ER_SQL_TYPE_MISMATCH,
-						 mem_str(pIn3),
-						 "numeric");
-					goto abort_due_to_error;
-				}
-			}
-			/* Handle the common case of integer comparison here, as an
-			 * optimization, to avoid a call to sqlMemCompare()
-			 */
-			if (mem_is_integer(pIn1) && mem_is_integer(pIn3)) {
-				if (!mem_is_unsigned(pIn1) &&
-				    !mem_is_unsigned(pIn3)) {
-					if (pIn3->u.i > pIn1->u.i)
-						res = +1;
-					else if (pIn3->u.i < pIn1->u.i)
-						res = -1;
-					else
-						res = 0;
-					goto compare_op;
-				}
-				if (mem_is_unsigned(pIn1) &&
-				    mem_is_unsigned(pIn3)) {
-					if (pIn3->u.u > pIn1->u.u)
-						res = +1;
-					else if (pIn3->u.u < pIn1->u.u)
-						res = -1;
-					else
-						res = 0;
-					goto compare_op;
-				}
-				if (mem_is_unsigned(pIn1) &&
-				    !mem_is_unsigned(pIn3)) {
-					res = -1;
-					goto compare_op;
-				}
-				res = 1;
-				goto compare_op;
-			}
-		} else if (type == FIELD_TYPE_STRING) {
-			if (!mem_is_string(pIn1) && mem_is_number(pIn1)) {
-				sqlVdbeMemStringify(pIn1);
-				flags1 = (pIn1->flags & ~MEM_TypeMask) | (flags1 & MEM_TypeMask);
-				assert(pIn1!=pIn3);
-			}
-			if (!mem_is_string(pIn3) && mem_is_number(pIn3)) {
-				sqlVdbeMemStringify(pIn3);
-				flags3 = (pIn3->flags & ~MEM_TypeMask) | (flags3 & MEM_TypeMask);
-			}
-		}
-		assert(pOp->p4type==P4_COLLSEQ || pOp->p4.pColl==0);
-		res = sqlMemCompare(pIn3, pIn1, pOp->p4.pColl);
+		if (mem_compare(pIn3, pIn1, &res, type, pOp->p4.pColl))
+			goto abort_due_to_error;
 	}
-			compare_op:
 	switch( pOp->opcode) {
 	case OP_Eq:    res2 = res==0;     break;
 	case OP_Ne:    res2 = res;        break;
@@ -1673,14 +1593,6 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 	default:       res2 = res>=0;     break;
 	}
 
-	/* Undo any changes made by mem_apply_type() to the input registers. */
-	assert((pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn));
-	pIn1->flags = flags1;
-	pIn1->field_type = ft_p1;
-	assert((pIn3->flags & MEM_Dyn) == (flags3 & MEM_Dyn));
-	pIn3->flags = flags3;
-	pIn3->field_type = ft_p3;
-
 	if (pOp->p5 & SQL_STOREP2) {
 		iCompare = res;
 		res2 = res2!=0;  /* For this path res2 must be exactly 0 or 1 */
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 20/53] sql: introduce mem_bitwise()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (18 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 19/53] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:03   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 21/53] sql: introduce mem_bit_not() Mergen Imeev via Tarantool-patches
                   ` (32 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_bitwise(). Function mem_bitwise() executes
bitwise operations with two operands and writes the result to the third
MEM.

Part of #5818
---
 src/box/sql/mem.c  | 51 +++++++++++++++++++++++++++++++++++++++++++
 src/box/sql/mem.h  |  3 +++
 src/box/sql/vdbe.c | 54 +++-------------------------------------------
 3 files changed, 57 insertions(+), 51 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 8119644ed..2b455e39f 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -556,6 +556,57 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
 	return 0;
 }
 
+int
+mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
+{
+	sqlVdbeMemSetNull(result);
+	result->field_type = FIELD_TYPE_INTEGER;
+	if (((left->flags | right->flags) & MEM_Null) != 0)
+		return 0;
+	int64_t l;
+	int64_t r;
+	bool unused;
+	if (sqlVdbeIntValue(left, &l, &unused) != 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(left), "integer");
+		return -1;
+	}
+	if (sqlVdbeIntValue(right, &r, &unused) != 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(right), "integer");
+		return -1;
+	}
+	if (op == OP_BitAnd) {
+		result->u.i = l & r;
+		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		return 0;
+	}
+	if (op == OP_BitOr) {
+		result->u.i = l | r;
+		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		return 0;
+	}
+	assert(op == OP_ShiftRight || op == OP_ShiftLeft);
+	bool is_left_shift = op == OP_ShiftLeft;
+	if (r < 0) {
+		is_left_shift = !is_left_shift;
+		r = r < -64 ? 64 : -r;
+	}
+	if (r >= 64) {
+		result->u.i = is_left_shift || l >= 0 ? 0 : -1;
+		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		return 0;
+	}
+	if (is_left_shift) {
+		result->u.i = l << r;
+		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		return 0;
+	}
+	result->u.i = l >> r;
+	result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+	return 0;
+}
+
 static int
 compare_blobs(const struct Mem *left, const struct Mem *right, int *result)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 096ce6533..8971e3d59 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -196,6 +196,9 @@ int
 mem_arithmetic(const struct Mem *left, const struct Mem *right,
 	       struct Mem *result, int op);
 
+int
+mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op);
+
 int
 mem_compare(const struct Mem *left, const struct Mem *right, int *result,
 	    enum field_type type, struct coll *coll);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 726066c17..cdcf7b2e8 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1304,60 +1304,12 @@ case OP_BitAnd:                 /* same as TK_BITAND, in1, in2, out3 */
 case OP_BitOr:                  /* same as TK_BITOR, in1, in2, out3 */
 case OP_ShiftLeft:              /* same as TK_LSHIFT, in1, in2, out3 */
 case OP_ShiftRight: {           /* same as TK_RSHIFT, in1, in2, out3 */
-	i64 iA;
-	u64 uA;
-	i64 iB;
-	u8 op;
-
 	pIn1 = &aMem[pOp->p1];
 	pIn2 = &aMem[pOp->p2];
-	pOut = vdbe_prepare_null_out(p, pOp->p3);
-	if (mem_is_null(pIn1) || mem_is_null(pIn2)) {
-		/* Force NULL be of type INTEGER. */
-		pOut->field_type = FIELD_TYPE_INTEGER;
-		break;
-	}
-	bool unused;
-	if (sqlVdbeIntValue(pIn2, (int64_t *) &iA, &unused) != 0) {
-		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 mem_str(pIn2), "integer");
-		goto abort_due_to_error;
-	}
-	if (sqlVdbeIntValue(pIn1, (int64_t *) &iB, &unused) != 0) {
-		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 mem_str(pIn1), "integer");
+	pOut = &aMem[pOp->p3];
+	if (mem_bitwise(pIn2, pIn1, pOut, pOp->opcode) != 0)
 		goto abort_due_to_error;
-	}
-	op = pOp->opcode;
-	if (op==OP_BitAnd) {
-		iA &= iB;
-	} else if (op==OP_BitOr) {
-		iA |= iB;
-	} else if (iB!=0) {
-		assert(op==OP_ShiftRight || op==OP_ShiftLeft);
-
-		/* If shifting by a negative amount, shift in the other direction */
-		if (iB<0) {
-			assert(OP_ShiftRight==OP_ShiftLeft+1);
-			op = 2*OP_ShiftLeft + 1 - op;
-			iB = iB>(-64) ? -iB : 64;
-		}
-
-		if (iB>=64) {
-			iA = (iA>=0 || op==OP_ShiftLeft) ? 0 : -1;
-		} else {
-			memcpy(&uA, &iA, sizeof(uA));
-			if (op==OP_ShiftLeft) {
-				uA <<= iB;
-			} else {
-				uA >>= iB;
-				/* Sign-extend on a right shift of a negative number */
-				if (iA<0) uA |= ((((u64)0xffffffff)<<32)|0xffffffff) << (64-iB);
-			}
-			memcpy(&iA, &uA, sizeof(iA));
-		}
-	}
-	mem_set_i64(pOut, iA);
+	assert(pOut->field_type == FIELD_TYPE_INTEGER);
 	break;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 21/53] sql: introduce mem_bit_not()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (19 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 20/53] sql: introduce mem_bitwise() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 22/53] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches
                   ` (31 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_bit_not(). Function mem_bit_not() executes
bitwise not operation on the first MEM and writes the result to the
second MEM.

Part of #5818
---
 src/box/sql/mem.c  | 19 +++++++++++++++++++
 src/box/sql/mem.h  |  3 +++
 src/box/sql/vdbe.c | 14 ++------------
 3 files changed, 24 insertions(+), 12 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 2b455e39f..c84b3775d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -607,6 +607,25 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
 	return 0;
 }
 
+int
+mem_bit_not(struct Mem *mem, struct Mem *result)
+{
+	mem_clear(result);
+	result->field_type = FIELD_TYPE_INTEGER;
+	if ((mem->flags & MEM_Null) != 0)
+		return 0;
+	int64_t i;
+	bool unused;
+	if (sqlVdbeIntValue(mem, &i, &unused) != 0) {
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(mem), "integer");
+		return -1;
+	}
+	i = ~i;
+	mem_set_int(result, i, i < 0);
+	return 0;
+}
+
 static int
 compare_blobs(const struct Mem *left, const struct Mem *right, int *result)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8971e3d59..3561b49b1 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -199,6 +199,9 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
 int
 mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op);
 
+int
+mem_bit_not(struct Mem *mem, struct Mem *result);
+
 int
 mem_compare(const struct Mem *left, const struct Mem *right, int *result,
 	    enum field_type type, struct coll *coll);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index cdcf7b2e8..80497b7b2 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1792,18 +1792,8 @@ case OP_Not: {                /* same as TK_NOT, in1, out2 */
 case OP_BitNot: {             /* same as TK_BITNOT, in1, out2 */
 	pIn1 = &aMem[pOp->p1];
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	/* Force NULL be of type INTEGER. */
-	pOut->field_type = FIELD_TYPE_INTEGER;
-	if (!mem_is_null(pIn1)) {
-		int64_t i;
-		bool is_neg;
-		if (sqlVdbeIntValue(pIn1, &i, &is_neg) != 0) {
-			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-				 mem_str(pIn1), "integer");
-			goto abort_due_to_error;
-		}
-		mem_set_i64(pOut, ~i);
-	}
+	if (mem_bit_not(pIn1, pOut) != 0)
+		goto abort_due_to_error;
 	break;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 22/53] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (20 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 21/53] sql: introduce mem_bit_not() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 23/53] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches
                   ` (30 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch adds initialization for newly created MEM objects in
sqlVdbeAllocUnpackedRecord(). Changing a MEM without being
initialized may give us unexpected result.

Part of #5818
---
 src/box/sql/vdbeaux.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index e58526401..9d07358e6 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2240,6 +2240,8 @@ sqlVdbeAllocUnpackedRecord(struct sql *db, struct key_def *key_def)
 	if (!p)
 		return 0;
 	p->aMem = (Mem *) & ((char *)p)[ROUND8(sizeof(UnpackedRecord))];
+	for (uint32_t i = 0; i < key_def->part_count + 1; ++i)
+		mem_create(&p->aMem[i]);
 	p->key_def = key_def;
 	p->nField = key_def->part_count + 1;
 	return p;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 23/53] sql: introduce mem_set_null()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (21 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 22/53] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 24/53] sql: introduce mem_set_integer() Mergen Imeev via Tarantool-patches
                   ` (29 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_null(). Function mem_set_null() clears MEM
and sets it to NULL.

Part of #5818
---
 src/box/sql/func.c    |  4 ++--
 src/box/sql/mem.c     | 36 +++++++++---------------------------
 src/box/sql/mem.h     |  6 +++---
 src/box/sql/vdbe.c    | 14 ++++++--------
 src/box/sql/vdbeapi.c |  4 ++--
 5 files changed, 22 insertions(+), 42 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 6b6081150..b61de18d8 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -290,7 +290,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
 				goto error;
 			break;
 		case MP_NIL:
-			sqlVdbeMemSetNull(&val[i]);
+			mem_set_null(&val[i]);
 			break;
 		default:
 			diag_set(ClientError, ER_SQL_EXECUTE,
@@ -361,7 +361,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
 				goto error;
 			break;
 		case MP_NIL:
-			sqlVdbeMemSetNull(&val[i]);
+			mem_set_null(&val[i]);
 			break;
 		default:
 			diag_set(ClientError, ER_SQL_EXECUTE,
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index c84b3775d..da2aa5c94 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -257,6 +257,12 @@ mem_destroy(struct Mem *mem)
 	mem->zMalloc = NULL;
 }
 
+void
+mem_set_null(struct Mem *mem)
+{
+	mem_clear(mem);
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -323,7 +329,7 @@ int
 mem_concat(struct Mem *left, struct Mem *right, struct Mem *result)
 {
 	assert(result != right);
-	sqlVdbeMemSetNull(result);
+	mem_clear(result);
 	result->field_type = field_type_MAX;
 
 	if (((left->flags | right->flags) & MEM_Null) != 0) {
@@ -374,7 +380,7 @@ int
 mem_arithmetic(const struct Mem *left, const struct Mem *right,
 	       struct Mem *result, int op)
 {
-	sqlVdbeMemSetNull(result);
+	mem_clear(result);
 	result->field_type = FIELD_TYPE_NUMBER;
 	if (((left->flags | right->flags) & MEM_Null) != 0)
 		return 0;
@@ -559,7 +565,7 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
 int
 mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
 {
-	sqlVdbeMemSetNull(result);
+	mem_clear(result);
 	result->field_type = FIELD_TYPE_INTEGER;
 	if (((left->flags | right->flags) & MEM_Null) != 0)
 		return 0;
@@ -2033,24 +2039,6 @@ sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
 	return 0;
 }
 
-/*
- * Delete any previous value and set the value stored in *pMem to NULL.
- *
- * This routine calls the Mem.xDel destructor to dispose of values that
- * require the destructor.  But it preserves the Mem.zMalloc memory allocation.
- * To free all resources, use mem_destroy(), which both calls this
- * routine to invoke the destructor and deallocates Mem.zMalloc.
- *
- * Use this routine to reset the Mem prior to insert a new value.
- *
- * Use mem_destroy() to complete erase the Mem prior to abandoning it.
- */
-void
-sqlVdbeMemSetNull(Mem * pMem)
-{
-	mem_clear(pMem);
-}
-
 /*
  * Delete any previous value and set the value to be a BLOB of length
  * n containing all zeros.
@@ -2081,12 +2069,6 @@ sqlValueSetStr(sql_value * v,	/* Value to be set */
 		sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
 }
 
-void
-sqlValueSetNull(sql_value * p)
-{
-	sqlVdbeMemSetNull((Mem *) p);
-}
-
 /*
  * Free an sql_value object
  */
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 3561b49b1..dc0d4a284 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -163,6 +163,9 @@ mem_create(struct Mem *mem);
 void
 mem_destroy(struct Mem *mem);
 
+void
+mem_set_null(struct Mem *mem);
+
 /**
  * Copy content of MEM from one MEM to another. In case source MEM contains
  * string or binary and allocation type is not STATIC, this value is copied to
@@ -431,12 +434,9 @@ mem_set_double(struct Mem *mem, double value);
 int
 sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
 void
-sqlVdbeMemSetNull(struct Mem *);
-void
 sqlVdbeMemSetZeroBlob(struct Mem *, int);
 void sqlValueSetStr(struct Mem *, int, const void *,
 			void (*)(void *));
-void sqlValueSetNull(struct Mem *);
 void sqlValueFree(struct Mem *);
 struct Mem *sqlValueNew(struct sql *);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 80497b7b2..f0b56033a 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -244,7 +244,7 @@ vdbe_prepare_null_out(struct Vdbe *v, int n)
 	assert(n <= (v->nMem + 1 - v->nCursor));
 	struct Mem *out = &v->aMem[n];
 	memAboutToChange(v, out);
-	sqlVdbeMemSetNull(out);
+	mem_set_null(out);
 	out->field_type = field_type_MAX;
 	return out;
 }
@@ -830,19 +830,17 @@ case OP_String: {          /* out2 */
  */
 case OP_Null: {           /* out2 */
 	int cnt;
-	u16 nullFlag;
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	cnt = pOp->p3-pOp->p2;
 	assert(pOp->p3<=(p->nMem+1 - p->nCursor));
-	pOut->flags = nullFlag = pOp->p1 ? (MEM_Null|MEM_Cleared) : MEM_Null;
-	pOut->n = 0;
+	if (pOp->p1 != 0)
+		pOut->flags = MEM_Null|MEM_Cleared;
 	while( cnt>0) {
 		pOut++;
 		memAboutToChange(p, pOut);
-		sqlVdbeMemSetNull(pOut);
-		pOut->flags = nullFlag;
-		pOut->field_type = field_type_MAX;
-		pOut->n = 0;
+		mem_set_null(pOut);
+		if (pOp->p1 != 0)
+			pOut->flags = MEM_Null|MEM_Cleared;
 		cnt--;
 	}
 	break;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 04016ea54..76af333e8 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -201,7 +201,7 @@ sql_result_bool(struct sql_context *ctx, bool value)
 void
 sql_result_null(sql_context * pCtx)
 {
-	sqlVdbeMemSetNull(pCtx->pOut);
+	mem_set_null(pCtx->pOut);
 }
 
 void
@@ -382,7 +382,7 @@ createAggContext(sql_context * p, int nByte)
 	Mem *pMem = p->pMem;
 	assert(!mem_is_aggregate(pMem));
 	if (nByte <= 0) {
-		sqlVdbeMemSetNull(pMem);
+		mem_set_null(pMem);
 		pMem->z = 0;
 	} else {
 		sqlVdbeMemClearAndResize(pMem, nByte);
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 24/53] sql: introduce mem_set_integer()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (22 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 23/53] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 25/53] sql: introduce mem_set_unsigned() Mergen Imeev via Tarantool-patches
                   ` (28 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_integer(). Function mem_set_integer()
clears MEM and sets it to given integer value.

Part of #5818
---
 src/box/sql/func.c    |  6 +--
 src/box/sql/mem.c     | 88 ++++++++++++++++++-------------------------
 src/box/sql/mem.h     | 21 ++++-------
 src/box/sql/vdbe.c    | 10 ++---
 src/box/sql/vdbeapi.c |  4 +-
 src/box/sql/vdbeaux.c |  6 +--
 6 files changed, 56 insertions(+), 79 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index b61de18d8..8f7550f30 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -278,7 +278,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
 			mem_set_double(&val[i], field.dval);
 			break;
 		case MP_INT:
-			mem_set_i64(&val[i], field.ival);
+			mem_set_integer(&val[i], field.ival, true);
 			break;
 		case MP_UINT:
 			mem_set_u64(&val[i], field.ival);
@@ -349,7 +349,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
 			mem_set_double(&val[i], mp_decode_double(&data));
 			break;
 		case MP_INT:
-			mem_set_i64(&val[i], mp_decode_int(&data));
+			mem_set_integer(&val[i], mp_decode_int(&data), true);
 			break;
 		case MP_UINT:
 			mem_set_u64(&val[i], mp_decode_uint(&data));
@@ -1989,7 +1989,7 @@ sumFinalize(sql_context * context)
 		} else if (p->approx) {
 			sql_result_double(context, p->rSum);
 		} else {
-			mem_set_int(context->pOut, p->iSum, p->is_neg);
+			mem_set_integer(context->pOut, p->iSum, p->is_neg);
 		}
 	}
 }
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index da2aa5c94..13a587aba 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -263,6 +263,15 @@ mem_set_null(struct Mem *mem)
 	mem_clear(mem);
 }
 
+void
+mem_set_integer(struct Mem *mem, int64_t value, bool is_neg)
+{
+	mem_clear(mem);
+	mem->u.i = value;
+	mem->flags = is_neg ? MEM_Int : MEM_UInt;
+	mem->field_type = FIELD_TYPE_INTEGER;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -557,8 +566,7 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
 	default:
 		unreachable();
 	}
-	result->u.i = ires;
-	result->flags = is_res_neg ? MEM_Int : MEM_UInt;
+	mem_set_integer(result, ires, is_res_neg);
 	return 0;
 }
 
@@ -571,6 +579,7 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
 		return 0;
 	int64_t l;
 	int64_t r;
+	int64_t res;
 	bool unused;
 	if (sqlVdbeIntValue(left, &l, &unused) != 0) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
@@ -583,13 +592,13 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
 		return -1;
 	}
 	if (op == OP_BitAnd) {
-		result->u.i = l & r;
-		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		res = l & r;
+		mem_set_integer(result, res, res < 0);
 		return 0;
 	}
 	if (op == OP_BitOr) {
-		result->u.i = l | r;
-		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		res = l | r;
+		mem_set_integer(result, res, res < 0);
 		return 0;
 	}
 	assert(op == OP_ShiftRight || op == OP_ShiftLeft);
@@ -599,17 +608,17 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
 		r = r < -64 ? 64 : -r;
 	}
 	if (r >= 64) {
-		result->u.i = is_left_shift || l >= 0 ? 0 : -1;
-		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		res = is_left_shift || l >= 0 ? 0 : -1;
+		mem_set_integer(result, res, res < 0);
 		return 0;
 	}
 	if (is_left_shift) {
-		result->u.i = l << r;
-		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+		res = l << r;
+		mem_set_integer(result, res, res < 0);
 		return 0;
 	}
-	result->u.i = l >> r;
-	result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
+	res = l >> r;
+	mem_set_integer(result, res, res < 0);
 	return 0;
 }
 
@@ -628,7 +637,7 @@ mem_bit_not(struct Mem *mem, struct Mem *result)
 		return -1;
 	}
 	i = ~i;
-	mem_set_int(result, i, i < 0);
+	mem_set_integer(result, i, i < 0);
 	return 0;
 }
 
@@ -1328,7 +1337,7 @@ mem_apply_numeric_type(struct Mem *record)
 	int64_t integer_value;
 	bool is_neg;
 	if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
-		mem_set_int(record, integer_value, is_neg);
+		mem_set_integer(record, integer_value, is_neg);
 		return 0;
 	}
 	double float_value;
@@ -1359,15 +1368,14 @@ vdbe_mem_numerify(struct Mem *mem)
 	if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
 		return 0;
 	if ((mem->flags & MEM_Bool) != 0) {
-		mem->u.u = mem->u.b;
-		MemSetTypeFlag(mem, MEM_UInt);
+		mem_set_integer(mem, (int64_t)mem->u.b, false);
 		return 0;
 	}
 	assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
 	bool is_neg;
 	int64_t i;
 	if (sql_atoi64(mem->z, &i, &is_neg, mem->n) == 0) {
-		mem_set_int(mem, i, is_neg);
+		mem_set_integer(mem, i, is_neg);
 	} else {
 		double d;
 		if (sqlAtoF(mem->z, &d, mem->n) == 0)
@@ -1425,12 +1433,11 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 				return -1;
 			if (type == FIELD_TYPE_UNSIGNED && is_neg)
 				return -1;
-			mem_set_int(pMem, val, is_neg);
+			mem_set_integer(pMem, val, is_neg);
 			return 0;
 		}
 		if ((pMem->flags & MEM_Bool) != 0) {
-			pMem->u.u = pMem->u.b;
-			MemSetTypeFlag(pMem, MEM_UInt);
+			mem_set_integer(pMem, (int64_t)pMem->u.b, false);
 			return 0;
 		}
 		if ((pMem->flags & MEM_Real) != 0) {
@@ -1438,7 +1445,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 			if (sqlVdbeRealValue(pMem, &d) != 0)
 				return -1;
 			if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
-				mem_set_int(pMem, d, d <= -1);
+				mem_set_integer(pMem, d, d <= -1);
 				return 0;
 			}
 			if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
@@ -1494,7 +1501,7 @@ mem_apply_integer_type(Mem *pMem)
 	assert(EIGHT_BYTE_ALIGNMENT(pMem));
 
 	if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
-		mem_set_int(pMem, ix, pMem->u.r <= -1);
+		mem_set_integer(pMem, ix, pMem->u.r <= -1);
 	return rc;
 }
 
@@ -1643,8 +1650,10 @@ mem_apply_type(struct Mem *record, enum field_type type)
 			} else {
 				if (double_compare_nint64(d, INT64_MIN, 1) < 0)
 					return 0;
-				if ((double)(int64_t)d == d)
-					mem_set_int(record, (int64_t)d, true);
+				if ((double)(int64_t)d == d) {
+					mem_set_integer(record, (int64_t)d,
+							true);
+				}
 			}
 			return 0;
 		}
@@ -1653,7 +1662,7 @@ mem_apply_type(struct Mem *record, enum field_type type)
 			int64_t i;
 			if (sql_atoi64(record->z, &i, &is_neg, record->n) != 0)
 				return -1;
-			mem_set_int(record, i, is_neg);
+			mem_set_integer(record, i, is_neg);
 		}
 		if ((record->flags & MEM_Int) == MEM_Int) {
 			if (type == FIELD_TYPE_UNSIGNED)
@@ -1777,9 +1786,9 @@ mem_convert_to_integer(struct Mem *mem)
 	if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
 		return -1;
 	if (d < (double)INT64_MAX)
-		mem_set_int(mem, (int64_t) d, d < 0);
+		mem_set_integer(mem, (int64_t) d, d < 0);
 	else
-		mem_set_int(mem, (uint64_t) d, false);
+		mem_set_integer(mem, (uint64_t) d, false);
 	return 0;
 }
 
@@ -1898,16 +1907,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
 	mem->u.p = ptr;
 }
 
-void
-mem_set_i64(struct Mem *mem, int64_t value)
-{
-	mem_clear(mem);
-	mem->u.i = value;
-	int flag = value < 0 ? MEM_Int : MEM_UInt;
-	MemSetTypeFlag(mem, flag);
-	mem->field_type = FIELD_TYPE_INTEGER;
-}
-
 void
 mem_set_u64(struct Mem *mem, uint64_t value)
 {
@@ -1917,21 +1916,6 @@ mem_set_u64(struct Mem *mem, uint64_t value)
 	mem->field_type = FIELD_TYPE_UNSIGNED;
 }
 
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg)
-{
-	mem_clear(mem);
-	if (is_neg) {
-		assert(value < 0);
-		mem->u.i = value;
-		MemSetTypeFlag(mem, MEM_Int);
-	} else {
-		mem->u.u = value;
-		MemSetTypeFlag(mem, MEM_UInt);
-	}
-	mem->field_type = FIELD_TYPE_INTEGER;
-}
-
 void
 mem_set_double(struct Mem *mem, double value)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index dc0d4a284..0fd902274 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -166,6 +166,13 @@ mem_destroy(struct Mem *mem);
 void
 mem_set_null(struct Mem *mem);
 
+/**
+ * Set integer value. According to is_neg flag value is considered
+ * to be signed or unsigned.
+ */
+void
+mem_set_integer(struct Mem *mem, int64_t value, bool is_neg);
+
 /**
  * Copy content of MEM from one MEM to another. In case source MEM contains
  * string or binary and allocation type is not STATIC, this value is copied to
@@ -409,24 +416,10 @@ mem_set_bool(struct Mem *mem, bool value);
 void
 mem_set_ptr(struct Mem *mem, void *ptr);
 
-/**
- * Set integer value. Depending on its sign MEM_Int (in case
- * of negative value) or MEM_UInt flag is set.
- */
-void
-mem_set_i64(struct Mem *mem, int64_t value);
-
 /** Set unsigned value and MEM_UInt flag. */
 void
 mem_set_u64(struct Mem *mem, uint64_t value);
 
-/**
- * Set integer value. According to is_neg flag value is considered
- * to be signed or unsigned.
- */
-void
-mem_set_int(struct Mem *mem, int64_t value, bool is_neg);
-
 /** Set double value and MEM_Real flag. */
 void
 mem_set_double(struct Mem *mem, double value);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index f0b56033a..92845d66d 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -731,7 +731,7 @@ case OP_Halt: {
  */
 case OP_Integer: {         /* out2 */
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	mem_set_i64(pOut, pOp->p1);
+	mem_set_integer(pOut, pOp->p1, pOp->p1 < 0);
 	break;
 }
 
@@ -756,7 +756,7 @@ case OP_Bool: {         /* out2 */
 case OP_Int64: {           /* out2 */
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	assert(pOp->p4.pI64!=0);
-	mem_set_int(pOut, *pOp->p4.pI64, pOp->p4type == P4_INT64);
+	mem_set_integer(pOut, *pOp->p4.pI64, pOp->p4type == P4_INT64);
 	break;
 }
 
@@ -2708,7 +2708,7 @@ skip_truncate:
 	r.nField = (u16)nField;
 
 	if (int_field > 0)
-		mem_set_int(&aMem[int_field], iKey, is_neg);
+		mem_set_integer(&aMem[int_field], iKey, is_neg);
 
 	r.default_rc = ((1 & (oc - OP_SeekLT)) ? -1 : +1);
 	assert(oc!=OP_SeekGT || r.default_rc==-1);
@@ -2999,8 +2999,8 @@ case OP_FCopy: {     /* out2 */
 		assert(mem_is_integer(pIn1));
 
 		pOut = vdbe_prepare_null_out(p, pOp->p2);
-		mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
-		pOut->field_type = pIn1->field_type;
+		if (mem_copy(pOut, pIn1) != 0)
+			goto abort_due_to_error;
 	}
 	break;
 }
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 76af333e8..b0f1f202f 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -189,7 +189,7 @@ sql_result_uint(sql_context *ctx, uint64_t u_val)
 void
 sql_result_int(sql_context *ctx, int64_t val)
 {
-	mem_set_i64(ctx->pOut, val);
+	mem_set_integer(ctx->pOut, val, val < 0);
 }
 
 void
@@ -840,7 +840,7 @@ sql_bind_int64(sql_stmt * pStmt, int i, sql_int64 iValue)
 		return -1;
 	int rc = sql_bind_type(p, i, "integer");
 	assert(iValue < 0);
-	mem_set_int(&p->aVar[i - 1], iValue, true);
+	mem_set_integer(&p->aVar[i - 1], iValue, true);
 	return rc;
 }
 
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 9d07358e6..3d1f094b8 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1324,13 +1324,13 @@ sqlVdbeList(Vdbe * p)
 			}
 		}
 
-		mem_set_i64(pMem, pOp->p1);
+		mem_set_integer(pMem, pOp->p1, pOp->p1 < 0);
 		pMem++;
 
-		mem_set_i64(pMem, pOp->p2);
+		mem_set_integer(pMem, pOp->p2, pOp->p2 < 0);
 		pMem++;
 
-		mem_set_i64(pMem, pOp->p3);
+		mem_set_integer(pMem, pOp->p3, pOp->p3 < 0);
 		pMem++;
 
 		if (sqlVdbeMemClearAndResize(pMem, 256)) {
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 25/53] sql: introduce mem_set_unsigned()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (23 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 24/53] sql: introduce mem_set_integer() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 26/53] sql: introduce mem_set_boolean() Mergen Imeev via Tarantool-patches
                   ` (27 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_unsigned(). Function mem_set_unsigned()
clears MEM and sets it to given unsigned value.

Part of #5818
---
 src/box/sql/func.c    |  4 ++--
 src/box/sql/mem.c     | 24 ++++++++++++------------
 src/box/sql/mem.h     |  7 +++----
 src/box/sql/vdbe.c    | 22 +++++++++++-----------
 src/box/sql/vdbeapi.c |  4 ++--
 src/box/sql/vdbeaux.c |  2 +-
 6 files changed, 31 insertions(+), 32 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 8f7550f30..d524c3d1e 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -281,7 +281,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
 			mem_set_integer(&val[i], field.ival, true);
 			break;
 		case MP_UINT:
-			mem_set_u64(&val[i], field.ival);
+			mem_set_unsigned(&val[i], field.ival);
 			break;
 		case MP_STR:
 			if (sqlVdbeMemSetStr(&val[i], field.sval.data,
@@ -352,7 +352,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
 			mem_set_integer(&val[i], mp_decode_int(&data), true);
 			break;
 		case MP_UINT:
-			mem_set_u64(&val[i], mp_decode_uint(&data));
+			mem_set_unsigned(&val[i], mp_decode_uint(&data));
 			break;
 		case MP_STR:
 			str = mp_decode_str(&data, &len);
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 13a587aba..f7a1a9b8a 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -272,6 +272,15 @@ mem_set_integer(struct Mem *mem, int64_t value, bool is_neg)
 	mem->field_type = FIELD_TYPE_INTEGER;
 }
 
+void
+mem_set_unsigned(struct Mem *mem, uint64_t value)
+{
+	mem_clear(mem);
+	mem->u.u = value;
+	mem->flags = MEM_UInt;
+	mem->field_type = FIELD_TYPE_UNSIGNED;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1449,7 +1458,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 				return 0;
 			}
 			if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
-				mem_set_u64(pMem, d);
+				mem_set_unsigned(pMem, d);
 				return 0;
 			}
 			return -1;
@@ -1646,7 +1655,7 @@ mem_apply_type(struct Mem *record, enum field_type type)
 							  1) > 0)
 					return 0;
 				if ((double)(uint64_t)d == d)
-					mem_set_u64(record, (uint64_t)d);
+					mem_set_unsigned(record, (uint64_t)d);
 			} else {
 				if (double_compare_nint64(d, INT64_MIN, 1) < 0)
 					return 0;
@@ -1765,7 +1774,7 @@ mem_convert_to_unsigned(struct Mem *mem)
 	double d = mem->u.r;
 	if (d < 0.0 || d >= (double)UINT64_MAX)
 		return -1;
-	mem_set_u64(mem, (uint64_t) d);
+	mem_set_unsigned(mem, (uint64_t) d);
 	return 0;
 }
 
@@ -1907,15 +1916,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
 	mem->u.p = ptr;
 }
 
-void
-mem_set_u64(struct Mem *mem, uint64_t value)
-{
-	mem_clear(mem);
-	mem->u.u = value;
-	MemSetTypeFlag(mem, MEM_UInt);
-	mem->field_type = FIELD_TYPE_UNSIGNED;
-}
-
 void
 mem_set_double(struct Mem *mem, double value)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 0fd902274..8090fb9a0 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -173,6 +173,9 @@ mem_set_null(struct Mem *mem);
 void
 mem_set_integer(struct Mem *mem, int64_t value, bool is_neg);
 
+void
+mem_set_unsigned(struct Mem *mem, uint64_t value);
+
 /**
  * Copy content of MEM from one MEM to another. In case source MEM contains
  * string or binary and allocation type is not STATIC, this value is copied to
@@ -416,10 +419,6 @@ mem_set_bool(struct Mem *mem, bool value);
 void
 mem_set_ptr(struct Mem *mem, void *ptr);
 
-/** Set unsigned value and MEM_UInt flag. */
-void
-mem_set_u64(struct Mem *mem, uint64_t value);
-
 /** Set double value and MEM_Real flag. */
 void
 mem_set_double(struct Mem *mem, double value);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 92845d66d..dd196d299 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -577,7 +577,7 @@ case OP_Gosub: {            /* jump */
 	pIn1 = &aMem[pOp->p1];
 	assert(VdbeMemDynamic(pIn1)==0);
 	memAboutToChange(p, pIn1);
-	mem_set_u64(pIn1, pOp - aOp);
+	mem_set_unsigned(pIn1, pOp - aOp);
 	REGISTER_TRACE(p, pOp->p1, pIn1);
 
 	/* Most jump operations do a goto to this spot in order to update
@@ -618,7 +618,7 @@ case OP_InitCoroutine: {     /* jump */
 	assert(pOp->p3>0 && pOp->p3<p->nOp);
 	pOut = &aMem[pOp->p1];
 	assert(!VdbeMemDynamic(pOut));
-	mem_set_u64(pOut, pOp->p3 - 1);
+	mem_set_unsigned(pOut, pOp->p3 - 1);
 	if (pOp->p2) goto jump_to_p2;
 	break;
 }
@@ -661,7 +661,7 @@ case OP_Yield: {            /* in1, jump */
 	pIn1 = &aMem[pOp->p1];
 	assert(VdbeMemDynamic(pIn1)==0);
 	int pcDest = (int)pIn1->u.u;
-	mem_set_u64(pIn1, pOp - aOp);
+	mem_set_unsigned(pIn1, pOp - aOp);
 	REGISTER_TRACE(p, pOp->p1, pIn1);
 	pOp = &aOp[pcDest];
 	break;
@@ -2142,7 +2142,7 @@ case OP_Count: {         /* out2 */
 		nEntry = tarantoolsqlEphemeralCount(pCrsr);
 	}
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	mem_set_u64(pOut, nEntry);
+	mem_set_unsigned(pOut, nEntry);
 	break;
 }
 
@@ -2920,7 +2920,7 @@ case OP_Sequence: {           /* out2 */
 	assert(p->apCsr[pOp->p1]!=0);
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	int64_t seq_val = p->apCsr[pOp->p1]->seqCount++;
-	mem_set_u64(pOut, seq_val);
+	mem_set_unsigned(pOut, seq_val);
 	break;
 }
 
@@ -2936,7 +2936,7 @@ case OP_NextSequenceId: {
 	uint64_t id = 0;
 	tarantoolSqlNextSeqId(&id);
 	id++;
-	mem_set_u64(pOut, id);
+	mem_set_unsigned(pOut, id);
 	break;
 }
 
@@ -2966,7 +2966,7 @@ case OP_NextIdEphemeral: {
 		goto abort_due_to_error;
 	}
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	mem_set_u64(pOut, rowid);
+	mem_set_unsigned(pOut, rowid);
 	break;
 }
 
@@ -4234,7 +4234,7 @@ case OP_OffsetLimit: {    /* in1, out2, in3 */
 			"values should not result in integer overflow");
 		goto abort_due_to_error;
 	}
-	mem_set_u64(pOut, x);
+	mem_set_unsigned(pOut, x);
 	break;
 }
 
@@ -4490,10 +4490,10 @@ case OP_Init: {          /* jump */
 case OP_IncMaxid: {
 	assert(pOp->p1 > 0);
 	pOut = vdbe_prepare_null_out(p, pOp->p1);
-
-	if (tarantoolsqlIncrementMaxid(&pOut->u.u) != 0)
+	uint64_t u;
+	if (tarantoolsqlIncrementMaxid(&u) != 0)
 		goto abort_due_to_error;
-	pOut->flags = MEM_UInt;
+	mem_set_unsigned(pOut, u);
 	break;
 }
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index b0f1f202f..0f41db21d 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -183,7 +183,7 @@ sql_result_double(sql_context * pCtx, double rVal)
 void
 sql_result_uint(sql_context *ctx, uint64_t u_val)
 {
-	mem_set_u64(ctx->pOut, u_val);
+	mem_set_unsigned(ctx->pOut, u_val);
 }
 
 void
@@ -851,7 +851,7 @@ sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value)
 	if (vdbeUnbind(p, i) != 0)
 		return -1;
 	int rc = sql_bind_type(p, i, "integer");
-	mem_set_u64(&p->aVar[i - 1], value);
+	mem_set_unsigned(&p->aVar[i - 1], value);
 	return rc;
 }
 
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 3d1f094b8..c4cf1109f 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1291,7 +1291,7 @@ sqlVdbeList(Vdbe * p)
 		}
 		if (p->explain == 1) {
 			assert(i >= 0);
-			mem_set_u64(pMem, i);
+			mem_set_unsigned(pMem, i);
 
 			pMem++;
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 26/53] sql: introduce mem_set_boolean()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (24 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 25/53] sql: introduce mem_set_unsigned() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 27/53] sql: refactor mem_set_double() Mergen Imeev via Tarantool-patches
                   ` (26 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_boolean(). Function mem_set_boolean()
clears MEM and sets it to given unsigned value.

Part of #5818
---
 src/box/sql/func.c    |  4 ++--
 src/box/sql/mem.c     | 26 +++++++++++++-------------
 src/box/sql/mem.h     |  6 +++---
 src/box/sql/vdbe.c    | 13 +++++++------
 src/box/sql/vdbeapi.c |  4 ++--
 5 files changed, 27 insertions(+), 26 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index d524c3d1e..6f5fe8cb6 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -269,7 +269,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
 		}
 		switch (field.type) {
 		case MP_BOOL:
-			mem_set_bool(&val[i], field.bval);
+			mem_set_boolean(&val[i], field.bval);
 			break;
 		case MP_FLOAT:
 			mem_set_double(&val[i], field.fval);
@@ -340,7 +340,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
 		const char *str;
 		switch (mp_typeof(*data)) {
 		case MP_BOOL:
-			mem_set_bool(&val[i], mp_decode_bool(&data));
+			mem_set_boolean(&val[i], mp_decode_bool(&data));
 			break;
 		case MP_FLOAT:
 			mem_set_double(&val[i], mp_decode_float(&data));
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index f7a1a9b8a..42a65d714 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -281,6 +281,15 @@ mem_set_unsigned(struct Mem *mem, uint64_t value)
 	mem->field_type = FIELD_TYPE_UNSIGNED;
 }
 
+void
+mem_set_boolean(struct Mem *mem, bool value)
+{
+	mem_clear(mem);
+	mem->u.b = value;
+	mem->flags = MEM_Bool;
+	mem->field_type = FIELD_TYPE_BOOLEAN;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1412,22 +1421,22 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 		return 0;
 	case FIELD_TYPE_BOOLEAN:
 		if ((pMem->flags & MEM_Int) != 0) {
-			mem_set_bool(pMem, pMem->u.i);
+			mem_set_boolean(pMem, pMem->u.i);
 			return 0;
 		}
 		if ((pMem->flags & MEM_UInt) != 0) {
-			mem_set_bool(pMem, pMem->u.u);
+			mem_set_boolean(pMem, pMem->u.u);
 			return 0;
 		}
 		if ((pMem->flags & MEM_Real) != 0) {
-			mem_set_bool(pMem, pMem->u.r);
+			mem_set_boolean(pMem, pMem->u.r);
 			return 0;
 		}
 		if ((pMem->flags & MEM_Str) != 0) {
 			bool value;
 			if (str_cast_to_boolean(pMem->z, &value) != 0)
 				return -1;
-			mem_set_bool(pMem, value);
+			mem_set_boolean(pMem, value);
 			return 0;
 		}
 		if ((pMem->flags & MEM_Bool) != 0)
@@ -1899,15 +1908,6 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
 	return 0;
 }
 
-void
-mem_set_bool(struct Mem *mem, bool value)
-{
-	mem_clear(mem);
-	mem->u.b = value;
-	mem->flags = MEM_Bool;
-	mem->field_type = FIELD_TYPE_BOOLEAN;
-}
-
 void
 mem_set_ptr(struct Mem *mem, void *ptr)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8090fb9a0..bf3690b7c 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -176,6 +176,9 @@ mem_set_integer(struct Mem *mem, int64_t value, bool is_neg);
 void
 mem_set_unsigned(struct Mem *mem, uint64_t value);
 
+void
+mem_set_boolean(struct Mem *mem, bool value);
+
 /**
  * Copy content of MEM from one MEM to another. In case source MEM contains
  * string or binary and allocation type is not STATIC, this value is copied to
@@ -408,9 +411,6 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type);
 int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
 int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
 
-void
-mem_set_bool(struct Mem *mem, bool value);
-
 /**
  * Set VDBE memory register with given pointer as a data.
  * @param mem VDBE memory register to update.
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index dd196d299..a7b7c5011 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -743,7 +743,7 @@ case OP_Integer: {         /* out2 */
 case OP_Bool: {         /* out2 */
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
 	assert(pOp->p1 == 1 || pOp->p1 == 0);
-	mem_set_bool(pOut, pOp->p1);
+	mem_set_boolean(pOut, pOp->p1);
 	break;
 }
 
@@ -1104,7 +1104,7 @@ case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
 case OP_CollSeq: {
 	assert(pOp->p4type==P4_COLLSEQ || pOp->p4.pColl == NULL);
 	if (pOp->p1) {
-		mem_set_bool(&aMem[pOp->p1], false);
+		mem_set_boolean(&aMem[pOp->p1], false);
 	}
 	break;
 }
@@ -1564,7 +1564,7 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
 			if ((pOp->opcode==OP_Eq)==res2) break;
 		}
 		pOut = vdbe_prepare_null_out(p, pOp->p2);
-		mem_set_bool(pOut, res2);
+		mem_set_boolean(pOut, res2);
 		REGISTER_TRACE(p, pOp->p2, pOut);
 	} else {
 		VdbeBranchTaken(res!=0, (pOp->p5 & SQL_NULLEQ)?2:3);
@@ -1754,7 +1754,7 @@ case OP_Or: {             /* same as TK_OR, in1, in2, out3 */
 	}
 	pOut = vdbe_prepare_null_out(p, pOp->p3);
 	if (v1 != 2)
-		mem_set_bool(pOut, v1);
+		mem_set_boolean(pOut, v1);
 	break;
 }
 
@@ -1775,7 +1775,7 @@ case OP_Not: {                /* same as TK_NOT, in1, out2 */
 				 mem_str(pIn1), "boolean");
 			goto abort_due_to_error;
 		}
-		mem_set_bool(pOut, ! pIn1->u.b);
+		mem_set_boolean(pOut, ! pIn1->u.b);
 	}
 	break;
 }
@@ -4365,7 +4365,8 @@ case OP_AggStep: {
 	if (pCtx->skipFlag) {
 		assert(pOp[-1].opcode==OP_CollSeq);
 		i = pOp[-1].p1;
-		if (i) mem_set_bool(&aMem[i], true);
+		if (i != 0)
+			mem_set_boolean(&aMem[i], true);
 	}
 	break;
 }
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 0f41db21d..d76efcfcf 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -195,7 +195,7 @@ sql_result_int(sql_context *ctx, int64_t val)
 void
 sql_result_bool(struct sql_context *ctx, bool value)
 {
-	mem_set_bool(ctx->pOut, value);
+	mem_set_boolean(ctx->pOut, value);
 }
 
 void
@@ -822,7 +822,7 @@ sql_bind_boolean(struct sql_stmt *stmt, int i, bool value)
 	if (vdbeUnbind(p, i) != 0)
 		return -1;
 	int rc = sql_bind_type(p, i, "boolean");
-	mem_set_bool(&p->aVar[i - 1], value);
+	mem_set_boolean(&p->aVar[i - 1], value);
 	return rc;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 27/53] sql: refactor mem_set_double()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (25 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 26/53] sql: introduce mem_set_boolean() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 28/53] sql: refactor mem_set_*_string() Mergen Imeev via Tarantool-patches
                   ` (25 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_double(). Function mem_set_double()
clears MEM and sets it to given unsigned value.

Part of #5818
---
 src/box/sql/mem.c | 22 +++++++++++-----------
 src/box/sql/mem.h |  7 +++----
 2 files changed, 14 insertions(+), 15 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 42a65d714..47a71fb30 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -290,6 +290,17 @@ mem_set_boolean(struct Mem *mem, bool value)
 	mem->field_type = FIELD_TYPE_BOOLEAN;
 }
 
+void
+mem_set_double(struct Mem *mem, double value)
+{
+	mem_clear(mem);
+	mem->field_type = FIELD_TYPE_DOUBLE;
+	if (sqlIsNaN(value))
+		return;
+	mem->u.r = value;
+	mem->flags = MEM_Real;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1916,17 +1927,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
 	mem->u.p = ptr;
 }
 
-void
-mem_set_double(struct Mem *mem, double value)
-{
-	mem_clear(mem);
-	if (sqlIsNaN(value))
-		return;
-	mem->u.r = value;
-	MemSetTypeFlag(mem, MEM_Real);
-	mem->field_type = FIELD_TYPE_DOUBLE;
-}
-
 /*
  * Change the value of a Mem to be a string or a BLOB.
  *
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index bf3690b7c..49b4e4b1a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -179,6 +179,9 @@ mem_set_unsigned(struct Mem *mem, uint64_t value);
 void
 mem_set_boolean(struct Mem *mem, bool value);
 
+void
+mem_set_double(struct Mem *mem, double value);
+
 /**
  * Copy content of MEM from one MEM to another. In case source MEM contains
  * string or binary and allocation type is not STATIC, this value is copied to
@@ -419,10 +422,6 @@ int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
 void
 mem_set_ptr(struct Mem *mem, void *ptr);
 
-/** Set double value and MEM_Real flag. */
-void
-mem_set_double(struct Mem *mem, double value);
-
 int
 sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
 void
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 28/53] sql: refactor mem_set_*_string()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (26 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 27/53] sql: refactor mem_set_double() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 29/53] sql: introduce mem_copy_string() Mergen Imeev via Tarantool-patches
                   ` (24 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_*_string() functions. Functions
mem_set_*_string() clears MEM and sets it to given string value.

Part of #5818
---
 src/box/sql/mem.c     | 111 +++++++++++++++++++++++++++++++++---------
 src/box/sql/mem.h     |  26 +++++++++-
 src/box/sql/sqlInt.h  |   4 --
 src/box/sql/vdbe.c    |   5 +-
 src/box/sql/vdbeapi.c |  47 ++++++++++++++----
 src/box/sql/vdbeaux.c |  45 +++++++----------
 6 files changed, 171 insertions(+), 67 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 47a71fb30..91ef7f3c8 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -301,6 +301,89 @@ mem_set_double(struct Mem *mem, double value)
 	mem->flags = MEM_Real;
 }
 
+static inline void
+mem_set_const_string(struct Mem *mem, char *value, uint32_t len, int alloc_type)
+{
+	assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
+	mem_clear(mem);
+	mem->z = value;
+	mem->n = len;
+	mem->flags = MEM_Str | alloc_type;
+	mem->field_type = FIELD_TYPE_STRING;
+}
+
+static inline void
+mem_set_dyn_string(struct Mem *mem, char *value, uint32_t len, int alloc_type)
+{
+	assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
+	assert(mem->szMalloc == 0 || value != mem->zMalloc);
+	assert(alloc_type == MEM_Dyn || alloc_type == 0);
+	mem_destroy(mem);
+	mem->z = value;
+	mem->n = len;
+	mem->flags = MEM_Str | alloc_type;
+	mem->field_type = FIELD_TYPE_STRING;
+	if (alloc_type == MEM_Dyn) {
+		mem->xDel = sql_free;
+	} else {
+		mem->xDel = NULL;
+		mem->zMalloc = mem->z;
+		mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
+	}
+}
+
+void
+mem_set_ephemeral_string(struct Mem *mem, char *value, uint32_t len)
+{
+	mem_set_const_string(mem, value, len, MEM_Ephem);
+}
+
+void
+mem_set_static_string(struct Mem *mem, char *value, uint32_t len)
+{
+	mem_set_const_string(mem, value, len, MEM_Static);
+}
+
+void
+mem_set_dynamic_string(struct Mem *mem, char *value, uint32_t len)
+{
+	mem_set_dyn_string(mem, value, len, MEM_Dyn);
+}
+
+void
+mem_set_allocated_string(struct Mem *mem, char *value, uint32_t len)
+{
+	mem_set_dyn_string(mem, value, len, 0);
+}
+
+void
+mem_set_ephemeral_string0(struct Mem *mem, char *value)
+{
+	mem_set_const_string(mem, value, strlen(value), MEM_Ephem);
+	mem->flags |= MEM_Term;
+}
+
+void
+mem_set_static_string0(struct Mem *mem, char *value)
+{
+	mem_set_const_string(mem, value, strlen(value), MEM_Static);
+	mem->flags |= MEM_Term;
+}
+
+void
+mem_set_dynamic_string0(struct Mem *mem, char *value)
+{
+	mem_set_dyn_string(mem, value, strlen(value), MEM_Dyn);
+	mem->flags |= MEM_Term;
+}
+
+void
+mem_set_allocated_string0(struct Mem *mem, char *value)
+{
+	mem_set_dyn_string(mem, value, strlen(value), 0);
+	mem->flags |= MEM_Term;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2039,20 +2122,6 @@ sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
 	pMem->z = 0;
 }
 
-/*
- * Change the string value of an sql_value object
- */
-void
-sqlValueSetStr(sql_value * v,	/* Value to be set */
-		   int n,	/* Length of string z */
-		   const void *z,	/* Text of the new string */
-		   void (*xDel) (void *)	/* Destructor for the string */
-    )
-{
-	if (v)
-		sqlVdbeMemSetStr((Mem *) v, z, n, 1, xDel);
-}
-
 /*
  * Free an sql_value object
  */
@@ -2780,13 +2849,9 @@ vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
 		break;
 	}
 	case MP_STR: {
-		/* XXX u32->int */
-		mem->n = (int) mp_decode_strl(&buf);
-		mem->flags = MEM_Str | MEM_Ephem;
-		mem->field_type = FIELD_TYPE_STRING;
-install_blob:
-		mem->z = (char *)buf;
-		buf += mem->n;
+		uint32_t len = mp_decode_strl(&buf);
+		mem_set_ephemeral_string(mem, (char *)buf, len);
+		buf += len;
 		break;
 	}
 	case MP_BIN: {
@@ -2794,7 +2859,9 @@ install_blob:
 		mem->n = (int) mp_decode_binl(&buf);
 		mem->flags = MEM_Blob | MEM_Ephem;
 		mem->field_type = FIELD_TYPE_VARBINARY;
-		goto install_blob;
+		mem->z = (char *)buf;
+		buf += mem->n;
+		break;
 	}
 	case MP_FLOAT: {
 		mem->u.r = mp_decode_float(&buf);
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 49b4e4b1a..70b5350f6 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -182,6 +182,30 @@ mem_set_boolean(struct Mem *mem, bool value);
 void
 mem_set_double(struct Mem *mem, double value);
 
+void
+mem_set_ephemeral_string(struct Mem *mem, char *value, uint32_t len);
+
+void
+mem_set_static_string(struct Mem *mem, char *value, uint32_t len);
+
+void
+mem_set_dynamic_string(struct Mem *mem, char *value, uint32_t len);
+
+void
+mem_set_allocated_string(struct Mem *mem, char *value, uint32_t len);
+
+void
+mem_set_ephemeral_string0(struct Mem *mem, char *value);
+
+void
+mem_set_static_string0(struct Mem *mem, char *value);
+
+void
+mem_set_dynamic_string0(struct Mem *mem, char *value);
+
+void
+mem_set_allocated_string0(struct Mem *mem, char *value);
+
 /**
  * Copy content of MEM from one MEM to another. In case source MEM contains
  * string or binary and allocation type is not STATIC, this value is copied to
@@ -426,8 +450,6 @@ int
 sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
 void
 sqlVdbeMemSetZeroBlob(struct Mem *, int);
-void sqlValueSetStr(struct Mem *, int, const void *,
-			void (*)(void *));
 void sqlValueFree(struct Mem *);
 struct Mem *sqlValueNew(struct sql *);
 
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 01c0cc5a1..f7ead1343 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -641,10 +641,6 @@ sql_bind_uint64(struct sql_stmt *stmt, int i, uint64_t value);
 int
 sql_bind_null(sql_stmt *, int);
 
-int
-sql_bind_text(sql_stmt *, int, const char *, int,
-		  void (*)(void *));
-
 int
 sql_bind_text64(sql_stmt *, int, const char *,
 		    sql_uint64, void (*)(void *));
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index a7b7c5011..341205cc4 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -809,9 +809,8 @@ case OP_String8: {         /* same as TK_STRING, out2 */
 case OP_String: {          /* out2 */
 	assert(pOp->p4.z!=0);
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	pOut->flags = MEM_Str|MEM_Static|MEM_Term;
-	pOut->z = pOp->p4.z;
-	pOut->n = pOp->p1;
+	assert(strlen(pOp->p4.z) == (size_t)pOp->p1);
+	mem_set_static_string0(pOut, pOp->p4.z);
 	UPDATE_MAX_BLOBSIZE(pOut);
 	break;
 }
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index d76efcfcf..5fc7b4e44 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -125,6 +125,27 @@ setResultStrOrError(sql_context * pCtx,	/* Function context */
 		    void (*xDel) (void *)	/* Destructor function */
     )
 {
+	if (xDel == SQL_STATIC) {
+		if (n < 0)
+			mem_set_static_string0(pCtx->pOut, (char *)z);
+		else
+			mem_set_static_string(pCtx->pOut, (char *)z, n);
+		return;
+	}
+	if (xDel == SQL_DYNAMIC) {
+		if (n < 0)
+			mem_set_allocated_string0(pCtx->pOut, (char *)z);
+		else
+			mem_set_allocated_string(pCtx->pOut, (char *)z, n);
+		return;
+	}
+	if (xDel != SQL_TRANSIENT) {
+		if (n < 0)
+			mem_set_dynamic_string0(pCtx->pOut, (char *)z);
+		else
+			mem_set_dynamic_string(pCtx->pOut, (char *)z, n);
+		return;
+	}
 	if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
 		pCtx->is_aborted = true;
 }
@@ -762,8 +783,24 @@ bindText(sql_stmt * pStmt,	/* The statement to bind against */
 	if (zData == NULL)
 		return 0;
 	pVar = &p->aVar[i - 1];
-	if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0)
+	if (xDel == SQL_STATIC) {
+		if (nData < 0)
+			mem_set_static_string0(pVar, (char *)zData);
+		else
+			mem_set_static_string(pVar, (char *)zData, nData);
+	} else if (xDel == SQL_DYNAMIC) {
+		if (nData < 0)
+			mem_set_allocated_string0(pVar, (char *)zData);
+		else
+			mem_set_allocated_string(pVar, (char *)zData, nData);
+	} else if (xDel != SQL_TRANSIENT) {
+		if (nData < 0)
+			mem_set_dynamic_string0(pVar, (char *)zData);
+		else
+			mem_set_dynamic_string(pVar, (char *)zData, nData);
+	} else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
 		return -1;
+	}
 	return sql_bind_type(p, i, "text");
 }
 
@@ -876,14 +913,6 @@ sql_bind_ptr(struct sql_stmt *stmt, int i, void *ptr)
 	return rc;
 }
 
-int
-sql_bind_text(sql_stmt * pStmt,
-		  int i, const char *zData, int nData, void (*xDel) (void *)
-    )
-{
-	return bindText(pStmt, i, zData, nData, xDel);
-}
-
 int
 sql_bind_text64(sql_stmt * pStmt,
 		    int i,
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index c4cf1109f..86da1449c 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1295,10 +1295,8 @@ sqlVdbeList(Vdbe * p)
 
 			pMem++;
 
-			pMem->flags = MEM_Static | MEM_Str | MEM_Term;
-			pMem->z = (char *)sqlOpcodeName(pOp->opcode);	/* Opcode */
-			assert(pMem->z != 0);
-			pMem->n = sqlStrlen30(pMem->z);
+			char *value = (char *)sqlOpcodeName(pOp->opcode);
+			mem_set_static_string0(pMem, value);
 			pMem++;
 
 			/* When an OP_Program opcode is encounter (the only opcode that has
@@ -1333,41 +1331,34 @@ sqlVdbeList(Vdbe * p)
 		mem_set_integer(pMem, pOp->p3, pOp->p3 < 0);
 		pMem++;
 
-		if (sqlVdbeMemClearAndResize(pMem, 256)) {
-			assert(p->db->mallocFailed);
+		char *buf = sqlDbMallocRaw(sql_get(), 256);
+		if (buf == NULL)
 			return -1;
-		}
-		pMem->flags = MEM_Str | MEM_Term;
-		zP4 = displayP4(pOp, pMem->z, pMem->szMalloc);
-
-		if (zP4 != pMem->z) {
-			pMem->n = 0;
-			sqlVdbeMemSetStr(pMem, zP4, -1, 1, 0);
+		zP4 = displayP4(pOp, buf, sqlDbMallocSize(sql_get(), buf));
+		if (zP4 != buf) {
+			sqlDbFree(sql_get(), buf);
+			mem_set_ephemeral_string0(pMem, zP4);
 		} else {
-			assert(pMem->z != 0);
-			pMem->n = sqlStrlen30(pMem->z);
+			mem_set_allocated_string0(pMem, zP4);
 		}
 		pMem++;
 
 		if (p->explain == 1) {
-			if (sqlVdbeMemClearAndResize(pMem, 4)) {
-				assert(p->db->mallocFailed);
+			buf = sqlDbMallocRaw(sql_get(), 4);
+			if (buf == NULL)
 				return -1;
-			}
-			pMem->flags = MEM_Str | MEM_Term;
-			pMem->n = 2;
-			sql_snprintf(3, pMem->z, "%.2x", pOp->p5);	/* P5 */
+			sql_snprintf(3, buf, "%.2x", pOp->p5);
+			mem_set_allocated_string0(pMem, buf);
 			pMem++;
 
 #ifdef SQL_ENABLE_EXPLAIN_COMMENTS
-			if (sqlVdbeMemClearAndResize(pMem, 500)) {
-				assert(p->db->mallocFailed);
+			buf = sqlDbMallocRaw(sql_get(), 500);
+			if (buf == NULL)
 				return -1;
-			}
-			pMem->flags = MEM_Str | MEM_Term;
-			pMem->n = displayComment(pOp, zP4, pMem->z, 500);
+			displayComment(pOp, zP4, buf, 500);
+			mem_set_allocated_string0(pMem, buf);
 #else
-			pMem->flags = MEM_Null;	/* Comment */
+			mem_set_null(pMem);
 #endif
 		}
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 29/53] sql: introduce mem_copy_string()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (27 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 28/53] sql: refactor mem_set_*_string() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:35 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 30/53] sql: introduce mem_set_*_binary() Mergen Imeev via Tarantool-patches
                   ` (23 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:35 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_copy_string() function. Function
mem_copy_string() clears MEM, allocates enough memory and copies given
string to allocated memory.

Part of #5818
---
 src/box/sql/func.c    |  8 +++-----
 src/box/sql/mem.c     | 30 ++++++++++++++++++++++++++++--
 src/box/sql/mem.h     |  6 ++++++
 src/box/sql/vdbeapi.c | 17 ++++++++++++++---
 4 files changed, 51 insertions(+), 10 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 6f5fe8cb6..1b3929259 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -284,9 +284,8 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
 			mem_set_unsigned(&val[i], field.ival);
 			break;
 		case MP_STR:
-			if (sqlVdbeMemSetStr(&val[i], field.sval.data,
-					     field.sval.len, 1,
-					     SQL_TRANSIENT) != 0)
+			if (mem_copy_string(&val[i], field.sval.data,
+					    field.sval.len) != 0)
 				goto error;
 			break;
 		case MP_NIL:
@@ -356,8 +355,7 @@ port_c_get_vdbemem(struct port *base, uint32_t *size)
 			break;
 		case MP_STR:
 			str = mp_decode_str(&data, &len);
-			if (sqlVdbeMemSetStr(&val[i], str, len,
-					     1, SQL_TRANSIENT) != 0)
+			if (mem_copy_string(&val[i], str, len) != 0)
 				goto error;
 			break;
 		case MP_NIL:
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 91ef7f3c8..59a378e1b 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -384,6 +384,32 @@ mem_set_allocated_string0(struct Mem *mem, char *value)
 	mem->flags |= MEM_Term;
 }
 
+int
+mem_copy_string(struct Mem *mem, const char *value, uint32_t len)
+{
+	bool is_own_value = (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
+			    mem->z == value;
+	if (sqlVdbeMemGrow(mem, len, is_own_value) != 0)
+		return -1;
+	if (!is_own_value)
+		memcpy(mem->z, value, len);
+	mem->n = len;
+	mem->flags = MEM_Str;
+	mem->field_type = FIELD_TYPE_STRING;
+	return 0;
+}
+
+int
+mem_copy_string0(struct Mem *mem, const char *value)
+{
+	uint32_t len = strlen(value);
+	if (mem_copy_string(mem, value, len + 1) != 0)
+		return -1;
+	mem->n = len;
+	mem->flags |= MEM_Term;
+	return 0;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1587,8 +1613,8 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 		assert(MEM_Str == (MEM_Blob >> 3));
 		if ((pMem->flags & MEM_Bool) != 0) {
 			const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
-			sqlVdbeMemSetStr(pMem, str_bool, strlen(str_bool), 1,
-					 SQL_TRANSIENT);
+			if (mem_copy_string0(pMem, str_bool) != 0)
+				return -1;
 			return 0;
 		}
 		pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 70b5350f6..12853606d 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -211,6 +211,12 @@ mem_set_allocated_string0(struct Mem *mem, char *value);
  * string or binary and allocation type is not STATIC, this value is copied to
  * newly allocated by destination MEM memory.
  */
+int
+mem_copy_string(struct Mem *mem, const char *value, uint32_t len);
+
+int
+mem_copy_string0(struct Mem *mem, const char *value);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 5fc7b4e44..5b5e5b0c8 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -146,7 +146,12 @@ setResultStrOrError(sql_context * pCtx,	/* Function context */
 			mem_set_dynamic_string(pCtx->pOut, (char *)z, n);
 		return;
 	}
-	if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 1, xDel) != 0)
+	if (n < 0) {
+		if (mem_copy_string0(pCtx->pOut, z) != 0)
+			pCtx->is_aborted = true;
+		return;
+	}
+	if (mem_copy_string(pCtx->pOut, z, n) != 0)
 		pCtx->is_aborted = true;
 }
 
@@ -798,8 +803,14 @@ bindText(sql_stmt * pStmt,	/* The statement to bind against */
 			mem_set_dynamic_string0(pVar, (char *)zData);
 		else
 			mem_set_dynamic_string(pVar, (char *)zData, nData);
-	} else if (sqlVdbeMemSetStr(pVar, zData, nData, 1, xDel) != 0) {
-		return -1;
+	} else {
+		if (nData < 0) {
+			if (mem_copy_string0(pVar, zData) != 0)
+				return -1;
+		} else {
+			if (mem_copy_string(pVar, zData, nData) != 0)
+				return -1;
+		}
 	}
 	return sql_bind_type(p, i, "text");
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 30/53] sql: introduce mem_set_*_binary()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (28 preceding siblings ...)
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 29/53] sql: introduce mem_copy_string() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 31/53] sql: introduce mem_copy_binary() Mergen Imeev via Tarantool-patches
                   ` (22 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_*_binary() functions. Functions
mem_set_*_binary() clears MEM and sets it to given binary value.

Part of #5818
---
 src/box/sql/mem.c     | 68 ++++++++++++++++++++++++++++++++++---------
 src/box/sql/mem.h     | 15 ++++++++--
 src/box/sql/vdbe.c    | 24 ++++++++++-----
 src/box/sql/vdbeapi.c | 21 +++++++++----
 4 files changed, 100 insertions(+), 28 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 59a378e1b..5ee49cdca 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -410,6 +410,61 @@ mem_copy_string0(struct Mem *mem, const char *value)
 	return 0;
 }
 
+static inline void
+mem_set_const_bin(struct Mem *mem, char *value, uint32_t size, int alloc_type)
+{
+	assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
+	mem_clear(mem);
+	mem->z = value;
+	mem->n = size;
+	mem->flags = MEM_Blob | alloc_type;
+	mem->field_type = FIELD_TYPE_VARBINARY;
+}
+
+static inline void
+mem_set_dyn_bin(struct Mem *mem, char *value, uint32_t size, int alloc_type)
+{
+	assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
+	assert(mem->szMalloc == 0 || value != mem->zMalloc);
+	assert(alloc_type == MEM_Dyn || alloc_type == 0);
+	mem_destroy(mem);
+	mem->z = value;
+	mem->n = size;
+	mem->flags = MEM_Blob | alloc_type;
+	mem->field_type = FIELD_TYPE_VARBINARY;
+	if (alloc_type == MEM_Dyn) {
+		mem->xDel = sql_free;
+	} else {
+		mem->xDel = NULL;
+		mem->zMalloc = mem->z;
+		mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
+	}
+}
+
+void
+mem_set_ephemeral_binary(struct Mem *mem, char *value, uint32_t size)
+{
+	mem_set_const_bin(mem, value, size, MEM_Ephem);
+}
+
+void
+mem_set_static_binary(struct Mem *mem, char *value, uint32_t size)
+{
+	mem_set_const_bin(mem, value, size, MEM_Static);
+}
+
+void
+mem_set_dynamic_binary(struct Mem *mem, char *value, uint32_t size)
+{
+	mem_set_dyn_bin(mem, value, size, MEM_Dyn);
+}
+
+void
+mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size)
+{
+	mem_set_dyn_bin(mem, value, size, 0);
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2596,19 +2651,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type)
 	return field_mp_plain_type_is_compatible(type, mp_type, true);
 }
 
-/* Allocate memory for internal VDBE structure on region. */
-int
-vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size)
-{
-	vdbe_mem->n = size;
-	vdbe_mem->z = region_alloc(&fiber()->gc, size);
-	if (vdbe_mem->z == NULL)
-		return -1;
-	vdbe_mem->flags = MEM_Ephem | MEM_Blob;
-	assert(sqlVdbeCheckMemInvariants(vdbe_mem));
-	return 0;
-}
-
 int
 sql_vdbemem_finalize(struct Mem *mem, struct func *func)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 12853606d..3d33fa98b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -217,6 +217,18 @@ mem_copy_string(struct Mem *mem, const char *value, uint32_t len);
 int
 mem_copy_string0(struct Mem *mem, const char *value);
 
+void
+mem_set_ephemeral_binary(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_static_binary(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_dynamic_binary(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
@@ -534,9 +546,6 @@ mem_is_type_compatible(struct Mem *mem, enum field_type type);
 
 /** MEM manipulate functions. */
 
-int
-vdbe_mem_alloc_blob_region(struct Mem *vdbe_mem, uint32_t size);
-
 /**
  * Memory cell mem contains the context of an aggregate function.
  * This routine calls the finalize method for that function. The
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 341205cc4..6ae77db63 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -854,8 +854,15 @@ case OP_Null: {           /* out2 */
 case OP_Blob: {                /* out2 */
 	assert(pOp->p1 <= SQL_MAX_LENGTH);
 	pOut = vdbe_prepare_null_out(p, pOp->p2);
-	sqlVdbeMemSetStr(pOut, pOp->p4.z, pOp->p1, 0, 0);
-	if (pOp->p3!=0) {
+	if (pOp->p3 == 0) {
+		/*
+		 * TODO: It is possible that vabinary should be stored as
+		 * ephemeral or static depending on value. There is no way to
+		 * determine right now, so it is stored as static.
+		 */
+		mem_set_static_binary(pOut, pOp->p4.z, pOp->p1);
+	} else {
+		sqlVdbeMemSetStr(pOut, pOp->p4.z, pOp->p1, 0, 0);
 		pOut->flags |= MEM_Subtype;
 		pOut->subtype = pOp->p3;
 	}
@@ -2110,9 +2117,7 @@ case OP_MakeRecord: {
 		 * sure previously allocated memory has gone.
 		 */
 		mem_destroy(pOut);
-		pOut->flags = MEM_Blob | MEM_Ephem;
-		pOut->n = tuple_size;
-		pOut->z = tuple;
+		mem_set_ephemeral_binary(pOut, tuple, tuple_size);
 	}
 	assert(sqlVdbeCheckMemInvariants(pOut));
 	assert(pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor));
@@ -3196,9 +3201,14 @@ case OP_RowData: {
 	}
 	testcase( n==0);
 
-	if (vdbe_mem_alloc_blob_region(pOut, n) != 0)
+	char *buf = region_alloc(&fiber()->gc, n);
+	if (buf == NULL) {
+		diag_set(OutOfMemory, n, "region_alloc", "buf");
 		goto abort_due_to_error;
-	sqlCursorPayload(pCrsr, 0, n, pOut->z);
+	}
+	sqlCursorPayload(pCrsr, 0, n, buf);
+	mem_set_ephemeral_binary(pOut, buf, n);
+	assert(sqlVdbeCheckMemInvariants(pOut));
 	UPDATE_MAX_BLOBSIZE(pOut);
 	REGISTER_TRACE(p, pOp->p2, pOut);
 	break;
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 5b5e5b0c8..0e51e4809 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -111,9 +111,8 @@ sql_metadata_is_full()
  * The following routines are used by user-defined functions to specify
  * the function result.
  *
- * The setStrOrError() function calls sqlVdbeMemSetStr() to store the
- * result as a string or blob but if the string or blob is too large, it
- * then sets the error code.
+ * The setStrOrError() function sets the result as a string or blob but
+ * if the string or blob is too large, it then sets the error code.
  *
  * The invokeValueDestructor(P,X) routine invokes destructor function X()
  * on value P is not going to be used and need to be destroyed.
@@ -183,7 +182,13 @@ sql_result_blob(sql_context * pCtx,
     )
 {
 	assert(n >= 0);
-	if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
+	if (xDel == SQL_STATIC)
+		mem_set_static_binary(pCtx->pOut, (char *)z, n);
+	else if (xDel == SQL_DYNAMIC)
+		mem_set_allocated_binary(pCtx->pOut, (char *)z, n);
+	else if (xDel != SQL_TRANSIENT)
+		mem_set_dynamic_binary(pCtx->pOut, (char *)z, n);
+	else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
 		pCtx->is_aborted = true;
 }
 
@@ -832,7 +837,13 @@ sql_bind_blob(sql_stmt * pStmt,
 	if (zData == NULL)
 		return 0;
 	struct Mem *var = &p->aVar[i - 1];
-	if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
+	if (xDel == SQL_STATIC)
+		mem_set_static_binary(var, (char *)zData, nData);
+	else if (xDel == SQL_DYNAMIC)
+		mem_set_allocated_binary(var, (char *)zData, nData);
+	else if (xDel != SQL_TRANSIENT)
+		mem_set_dynamic_binary(var, (char *)zData, nData);
+	else if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
 		return -1;
 	return sql_bind_type(p, i, "varbinary");
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 31/53] sql: introduce mem_copy_binary()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (29 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 30/53] sql: introduce mem_set_*_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 32/53] sql: introduce mem_set_zerobinary() Mergen Imeev via Tarantool-patches
                   ` (21 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_copy_binary() function. Function
mem_copy_binary() clears MEM, allocates enough memory and copies given
binary to allocated memory.

Part of #5818
---
 src/box/sql/mem.c      | 14 ++++++++++++++
 src/box/sql/mem.h      |  3 +++
 src/box/sql/vdbe.c     |  7 ++-----
 src/box/sql/vdbeapi.c  |  4 ++--
 src/box/sql/vdbesort.c |  6 +-----
 5 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 5ee49cdca..99beec9ad 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -465,6 +465,20 @@ mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size)
 	mem_set_dyn_bin(mem, value, size, 0);
 }
 
+int
+mem_copy_binary(struct Mem *mem, const char *value, uint32_t size)
+{
+	bool is_own_value = (mem->flags & MEM_Blob) != 0 && mem->z == value;
+	if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
+		return -1;
+	if (!is_own_value)
+		memcpy(mem->z, value, size);
+	mem->n = size;
+	mem->flags = MEM_Blob;
+	mem->field_type = FIELD_TYPE_VARBINARY;
+	return 0;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 3d33fa98b..f44bb9bcf 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -229,6 +229,9 @@ mem_set_dynamic_binary(struct Mem *mem, char *value, uint32_t size);
 void
 mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size);
 
+int
+mem_copy_binary(struct Mem *mem, const char *value, uint32_t size);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 6ae77db63..9fc08e30c 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2105,11 +2105,8 @@ case OP_MakeRecord: {
 	 * routine.
 	 */
 	if (bIsEphemeral) {
-		if (sqlVdbeMemClearAndResize(pOut, tuple_size) != 0)
+		if (mem_copy_binary(pOut, tuple, tuple_size) != 0)
 			goto abort_due_to_error;
-		pOut->flags = MEM_Blob;
-		pOut->n = tuple_size;
-		memcpy(pOut->z, tuple, tuple_size);
 		region_truncate(region, used);
 	} else {
 		/* Allocate memory on the region for the tuple
@@ -2449,7 +2446,7 @@ case OP_SequenceTest: {
  * Open a new cursor that points to a fake table that contains a single
  * row of data.  The content of that one row is the content of memory
  * register P2.  In other words, cursor P1 becomes an alias for the
- * MEM_Blob content contained in register P2.
+ * MEM with binary content contained in register P2.
  *
  * A pseudo-table created by this opcode is used to hold a single
  * row output from the sorter so that the row can be decomposed into
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 0e51e4809..299172554 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -188,7 +188,7 @@ sql_result_blob(sql_context * pCtx,
 		mem_set_allocated_binary(pCtx->pOut, (char *)z, n);
 	else if (xDel != SQL_TRANSIENT)
 		mem_set_dynamic_binary(pCtx->pOut, (char *)z, n);
-	else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
+	else if (mem_copy_binary(pCtx->pOut, z, n) != 0)
 		pCtx->is_aborted = true;
 }
 
@@ -843,7 +843,7 @@ sql_bind_blob(sql_stmt * pStmt,
 		mem_set_allocated_binary(var, (char *)zData, nData);
 	else if (xDel != SQL_TRANSIENT)
 		mem_set_dynamic_binary(var, (char *)zData, nData);
-	else if (sqlVdbeMemSetStr(var, zData, nData, 0, xDel) != 0)
+	else if (mem_copy_binary(var, zData, nData) != 0)
 		return -1;
 	return sql_bind_type(p, i, "varbinary");
 }
diff --git a/src/box/sql/vdbesort.c b/src/box/sql/vdbesort.c
index a9a5f45af..51fe691d2 100644
--- a/src/box/sql/vdbesort.c
+++ b/src/box/sql/vdbesort.c
@@ -2164,12 +2164,8 @@ sqlVdbeSorterRowkey(const VdbeCursor * pCsr, Mem * pOut)
 	assert(pCsr->eCurType == CURTYPE_SORTER);
 	pSorter = pCsr->uc.pSorter;
 	pKey = vdbeSorterRowkey(pSorter, &nKey);
-	if (sqlVdbeMemClearAndResize(pOut, nKey)) {
+	if (mem_copy_binary(pOut, pKey, nKey) != 0)
 		return -1;
-	}
-	pOut->n = nKey;
-	MemSetTypeFlag(pOut, MEM_Blob);
-	memcpy(pOut->z, pKey, nKey);
 
 	return 0;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 32/53] sql: introduce mem_set_zerobinary()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (30 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 31/53] sql: introduce mem_copy_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary() Mergen Imeev via Tarantool-patches
                   ` (20 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_zerobinary() function. Function
mem_set_zerobinary() clears MEM and sets it to binary value that
consists of given number of zero bytes.

Part of #5818
---
 src/box/sql/mem.c     | 29 +++++++++++++----------------
 src/box/sql/mem.h     |  5 +++--
 src/box/sql/vdbeapi.c |  6 +++---
 3 files changed, 19 insertions(+), 21 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 99beec9ad..63a4d0d3c 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -479,6 +479,19 @@ mem_copy_binary(struct Mem *mem, const char *value, uint32_t size)
 	return 0;
 }
 
+void
+mem_set_zerobinary(struct Mem *mem, int n)
+{
+	mem_destroy(mem);
+	if (n < 0)
+		n = 0;
+	mem->u.nZero = n;
+	mem->z = NULL;
+	mem->n = 0;
+	mem->flags = MEM_Blob | MEM_Zero;
+	mem->field_type = FIELD_TYPE_VARBINARY;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2201,22 +2214,6 @@ sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
 	return 0;
 }
 
-/*
- * Delete any previous value and set the value to be a BLOB of length
- * n containing all zeros.
- */
-void
-sqlVdbeMemSetZeroBlob(Mem * pMem, int n)
-{
-	mem_destroy(pMem);
-	pMem->flags = MEM_Blob | MEM_Zero;
-	pMem->n = 0;
-	if (n < 0)
-		n = 0;
-	pMem->u.nZero = n;
-	pMem->z = 0;
-}
-
 /*
  * Free an sql_value object
  */
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f44bb9bcf..4b34bba6c 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -229,6 +229,9 @@ mem_set_dynamic_binary(struct Mem *mem, char *value, uint32_t size);
 void
 mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size);
 
+void
+mem_set_zerobinary(struct Mem *mem, int n);
+
 int
 mem_copy_binary(struct Mem *mem, const char *value, uint32_t size);
 
@@ -469,8 +472,6 @@ mem_set_ptr(struct Mem *mem, void *ptr);
 
 int
 sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
-void
-sqlVdbeMemSetZeroBlob(struct Mem *, int);
 void sqlValueFree(struct Mem *);
 struct Mem *sqlValueNew(struct sql *);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 299172554..d6d6e54f5 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -266,7 +266,7 @@ sql_result_value(sql_context * pCtx, sql_value * pValue)
 void
 sql_result_zeroblob(sql_context * pCtx, int n)
 {
-	sqlVdbeMemSetZeroBlob(pCtx->pOut, n);
+	mem_set_zerobinary(pCtx->pOut, n);
 }
 
 int
@@ -278,7 +278,7 @@ sql_result_zeroblob64(sql_context * pCtx, u64 n)
 			 "is too big");
 		return -1;
 	}
-	sqlVdbeMemSetZeroBlob(pCtx->pOut, (int)n);
+	mem_set_zerobinary(pCtx->pOut, (int)n);
 	return 0;
 }
 
@@ -956,7 +956,7 @@ sql_bind_zeroblob(sql_stmt * pStmt, int i, int n)
 	Vdbe *p = (Vdbe *) pStmt;
 	if (vdbeUnbind(p, i) != 0)
 		return -1;
-	sqlVdbeMemSetZeroBlob(&p->aVar[i - 1], n);
+	mem_set_zerobinary(&p->aVar[i - 1], n);
 	return 0;
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (31 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 32/53] sql: introduce mem_set_zerobinary() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 34/53] sql: introduce mem_set_*_map() and mem_set_*_array() Mergen Imeev via Tarantool-patches
                   ` (19 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_append_to_binary() function. Function
mem_append_to_binary() receives MEM with binary value and append to it
another binary value. MEM is updated accordingly.

Part of #5818
---
 src/box/sql/mem.c     | 16 ++++++++++++++++
 src/box/sql/mem.h     |  3 +++
 src/box/sql/vdbeaux.c | 23 +++++++++++++++--------
 3 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 63a4d0d3c..7885caaf5 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -492,6 +492,22 @@ mem_set_zerobinary(struct Mem *mem, int n)
 	mem->field_type = FIELD_TYPE_VARBINARY;
 }
 
+int
+mem_append_to_binary(struct Mem *mem, const char *value, uint32_t size)
+{
+	if ((mem->flags & MEM_Blob) == 0) {
+		diag_set(ClientError, ER_INCONSISTENT_TYPES, "varbinary",
+			 mem_type_to_str(mem));
+		return -1;
+	}
+	assert(mem->field_type == FIELD_TYPE_VARBINARY);
+	if (sqlVdbeMemGrow(mem, mem->n + size, 1) != 0)
+		return -1;
+	memcpy(&mem->z[mem->n], value, size);
+	mem->n += size;
+	return 0;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 4b34bba6c..3bd04bbbd 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -235,6 +235,9 @@ mem_set_zerobinary(struct Mem *mem, int n);
 int
 mem_copy_binary(struct Mem *mem, const char *value, uint32_t size);
 
+int
+mem_append_to_binary(struct Mem *mem, const char *value, uint32_t size);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 86da1449c..2e8669138 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1305,19 +1305,26 @@ sqlVdbeList(Vdbe * p)
 			 * has not already been seen.
 			 */
 			if (pOp->p4type == P4_SUBPROGRAM) {
-				int nByte = (nSub + 1) * sizeof(SubProgram *);
 				int j;
 				for (j = 0; j < nSub; j++) {
 					if (apSub[j] == pOp->p4.pProgram)
 						break;
 				}
-				if (j == nSub &&
-				    sqlVdbeMemGrow(pSub, nByte,
-						   nSub != 0) == 0) {
-					apSub = (SubProgram **) pSub->z;
-					apSub[nSub++] = pOp->p4.pProgram;
-					pSub->flags |= MEM_Blob;
-					pSub->n = nSub * sizeof(SubProgram *);
+				if (j == nSub) {
+					uint32_t size = sizeof(SubProgram *);
+					char *value = (char *)&pOp->p4.pProgram;
+					if (nSub == 0) {
+						if (mem_copy_binary(pSub, value,
+								    size) != 0)
+							return -1;
+					} else {
+						assert(0);
+						if (mem_append_to_binary(pSub,
+									 value,
+									 size) != 0)
+							return -1;
+					}
+					++nSub;
 				}
 			}
 		}
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 34/53] sql: introduce mem_set_*_map() and mem_set_*_array()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (32 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 35/53] sql: introduce mem_set_undefined() Mergen Imeev via Tarantool-patches
                   ` (18 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_*_map() and mem_set_*_array() functions.
These function should be used to set MAP or ARRAY values to MEM.

Part of #5818
---
 src/box/sql/mem.c  | 176 +++++++++++++++++++++------------------------
 src/box/sql/mem.h  |  26 ++++++-
 src/box/sql/vdbe.c |   8 ++-
 3 files changed, 109 insertions(+), 101 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 7885caaf5..583de00a2 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -508,6 +508,86 @@ mem_append_to_binary(struct Mem *mem, const char *value, uint32_t size)
 	return 0;
 }
 
+void
+mem_set_ephemeral_map(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_MAP);
+	mem_set_const_bin(mem, value, size, MEM_Ephem);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_MAP;
+}
+
+void
+mem_set_static_map(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_MAP);
+	mem_set_const_bin(mem, value, size, MEM_Static);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_MAP;
+}
+
+void
+mem_set_dynamic_map(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_MAP);
+	mem_set_dyn_bin(mem, value, size, MEM_Dyn);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_MAP;
+}
+
+void
+mem_set_allocated_map(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_MAP);
+	mem_set_dyn_bin(mem, value, size, 0);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_MAP;
+}
+
+void
+mem_set_ephemeral_array(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_ARRAY);
+	mem_set_const_bin(mem, value, size, MEM_Ephem);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_ARRAY;
+}
+
+void
+mem_set_static_array(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_ARRAY);
+	mem_set_const_bin(mem, value, size, MEM_Static);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_ARRAY;
+}
+
+void
+mem_set_dynamic_array(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_ARRAY);
+	mem_set_dyn_bin(mem, value, size, MEM_Dyn);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_ARRAY;
+}
+
+void
+mem_set_allocated_array(struct Mem *mem, char *value, uint32_t size)
+{
+	assert(mp_typeof(*value) == MP_ARRAY);
+	mem_set_dyn_bin(mem, value, size, 0);
+	mem->flags |= MEM_Subtype;
+	mem->subtype = SQL_SUBTYPE_MSGPACK;
+	mem->field_type = FIELD_TYPE_ARRAY;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2134,102 +2214,6 @@ mem_set_ptr(struct Mem *mem, void *ptr)
 	mem->u.p = ptr;
 }
 
-/*
- * Change the value of a Mem to be a string or a BLOB.
- *
- * The memory management strategy depends on the value of the xDel
- * parameter. If the value passed is SQL_TRANSIENT, then the
- * string is copied into a (possibly existing) buffer managed by the
- * Mem structure. Otherwise, any existing buffer is freed and the
- * pointer copied.
- *
- * If the string is too large (if it exceeds the SQL_LIMIT_LENGTH
- * size limit) then no memory allocation occurs.  If the string can be
- * stored without allocating memory, then it is.  If a memory allocation
- * is required to store the string, then value of pMem is unchanged.  In
- * either case, error is returned.
- */
-int
-sqlVdbeMemSetStr(Mem * pMem,	/* Memory cell to set to string value */
-		     const char *z,	/* String pointer */
-		     int n,	/* Bytes in string, or negative */
-		     u8 not_blob,	/* Encoding of z.  0 for BLOBs */
-		     void (*xDel) (void *)	/* Destructor function */
-    )
-{
-	int nByte = n;		/* New value for pMem->n */
-	int iLimit;		/* Maximum allowed string or blob size */
-	u16 flags = 0;		/* New value for pMem->flags */
-
-	/* If z is a NULL pointer, set pMem to contain an SQL NULL. */
-	if (!z) {
-		mem_clear(pMem);
-		return 0;
-	}
-
-	if (pMem->db) {
-		iLimit = pMem->db->aLimit[SQL_LIMIT_LENGTH];
-	} else {
-		iLimit = SQL_MAX_LENGTH;
-	}
-	flags = (not_blob == 0 ? MEM_Blob : MEM_Str);
-	if (nByte < 0) {
-		assert(not_blob != 0);
-		nByte = sqlStrlen30(z);
-		if (nByte > iLimit)
-			nByte = iLimit + 1;
-		flags |= MEM_Term;
-	}
-
-	/* The following block sets the new values of Mem.z and Mem.xDel. It
-	 * also sets a flag in local variable "flags" to indicate the memory
-	 * management (one of MEM_Dyn or MEM_Static).
-	 */
-	if (xDel == SQL_TRANSIENT) {
-		int nAlloc = nByte;
-		if (flags & MEM_Term) {
-			nAlloc += 1; //SQL_UTF8
-		}
-		if (nByte > iLimit) {
-			diag_set(ClientError, ER_SQL_EXECUTE, "string or binary"\
-				 "string is too big");
-			return -1;
-		}
-		testcase(nAlloc == 0);
-		testcase(nAlloc == 31);
-		testcase(nAlloc == 32);
-		if (sqlVdbeMemClearAndResize(pMem, MAX(nAlloc, 32))) {
-			return -1;
-		}
-		memcpy(pMem->z, z, nAlloc);
-	} else if (xDel == SQL_DYNAMIC) {
-		mem_destroy(pMem);
-		pMem->zMalloc = pMem->z = (char *)z;
-		pMem->szMalloc = sqlDbMallocSize(pMem->db, pMem->zMalloc);
-	} else {
-		mem_destroy(pMem);
-		pMem->z = (char *)z;
-		pMem->xDel = xDel;
-		flags |= ((xDel == SQL_STATIC) ? MEM_Static : MEM_Dyn);
-	}
-
-	pMem->n = nByte;
-	pMem->flags = flags;
-	assert((pMem->flags & (MEM_Str | MEM_Blob)) != 0);
-	if ((pMem->flags & MEM_Str) != 0)
-		pMem->field_type = FIELD_TYPE_STRING;
-	else
-		pMem->field_type = FIELD_TYPE_VARBINARY;
-
-	if (nByte > iLimit) {
-		diag_set(ClientError, ER_SQL_EXECUTE, "string or binary string"\
-			 "is too big");
-		return -1;
-	}
-
-	return 0;
-}
-
 /*
  * Free an sql_value object
  */
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 3bd04bbbd..b94495f89 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -238,6 +238,30 @@ mem_copy_binary(struct Mem *mem, const char *value, uint32_t size);
 int
 mem_append_to_binary(struct Mem *mem, const char *value, uint32_t size);
 
+void
+mem_set_ephemeral_map(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_static_map(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_dynamic_map(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_allocated_map(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_ephemeral_array(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_static_array(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_dynamic_array(struct Mem *mem, char *value, uint32_t size);
+
+void
+mem_set_allocated_array(struct Mem *mem, char *value, uint32_t size);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
@@ -473,8 +497,6 @@ int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
 void
 mem_set_ptr(struct Mem *mem, void *ptr);
 
-int
-sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
 void sqlValueFree(struct Mem *);
 struct Mem *sqlValueNew(struct sql *);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 9fc08e30c..c91aeede3 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -862,9 +862,11 @@ case OP_Blob: {                /* out2 */
 		 */
 		mem_set_static_binary(pOut, pOp->p4.z, pOp->p1);
 	} else {
-		sqlVdbeMemSetStr(pOut, pOp->p4.z, pOp->p1, 0, 0);
-		pOut->flags |= MEM_Subtype;
-		pOut->subtype = pOp->p3;
+		assert(pOp->p3 == SQL_SUBTYPE_MSGPACK);
+		if (mp_typeof(*pOp->p4.z) == MP_MAP)
+			mem_set_static_map(pOut, pOp->p4.z, pOp->p1);
+		else
+			mem_set_static_array(pOut, pOp->p4.z, pOp->p1);
 	}
 	UPDATE_MAX_BLOBSIZE(pOut);
 	break;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 35/53] sql: introduce mem_set_undefined()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (33 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 34/53] sql: introduce mem_set_*_map() and mem_set_*_array() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 36/53] sql: introduce mem_set_pointer() Mergen Imeev via Tarantool-patches
                   ` (17 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_undefined() function. Function
mem_set_undefined() invalidates MEM. It does not clears MEM prior to
invalidating it.

Part of #5818
---
 src/box/sql/mem.c     | 7 +++++++
 src/box/sql/mem.h     | 3 +++
 src/box/sql/vdbe.c    | 8 ++++----
 src/box/sql/vdbeaux.c | 2 +-
 4 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 583de00a2..fd2710478 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -588,6 +588,13 @@ mem_set_allocated_array(struct Mem *mem, char *value, uint32_t size)
 	mem->field_type = FIELD_TYPE_ARRAY;
 }
 
+void
+mem_set_undefined(struct Mem *mem)
+{
+	mem->flags = MEM_Undefined;
+	mem->field_type = field_type_MAX;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index b94495f89..75486ba3a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -262,6 +262,9 @@ mem_set_dynamic_array(struct Mem *mem, char *value, uint32_t size);
 void
 mem_set_allocated_array(struct Mem *mem, char *value, uint32_t size);
 
+void
+mem_set_undefined(struct Mem *mem);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index c91aeede3..f899dfe7a 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -597,7 +597,7 @@ case OP_Return: {           /* in1 */
 	pIn1 = &aMem[pOp->p1];
 	assert(mem_is_unsigned(pIn1));
 	pOp = &aOp[pIn1->u.u];
-	pIn1->flags = MEM_Undefined;
+	mem_set_undefined(pIn1);
 	break;
 }
 
@@ -640,7 +640,7 @@ case OP_EndCoroutine: {           /* in1 */
 	assert(pCaller->opcode==OP_Yield);
 	assert(pCaller->p2>=0 && pCaller->p2<p->nOp);
 	pOp = &aOp[pCaller->p2 - 1];
-	pIn1->flags = MEM_Undefined;
+	mem_set_undefined(pIn1);
 	break;
 }
 
@@ -4091,8 +4091,8 @@ case OP_Program: {        /* jump */
 
 		pEnd = &VdbeFrameMem(pFrame)[pFrame->nChildMem];
 		for(pMem=VdbeFrameMem(pFrame); pMem!=pEnd; pMem++) {
-			pMem->flags = MEM_Undefined;
-			pMem->db = db;
+			mem_create(pMem);
+			mem_set_undefined(pMem);
 		}
 	} else {
 		pFrame = pRt->u.pFrame;
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 2e8669138..ccabf6035 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -1568,7 +1568,7 @@ sqlVdbeMakeReady(Vdbe * p,	/* The VDBE */
 		p->nMem = nMem;
 		for (int i = 0; i < nMem; ++i) {
 			mem_create(&p->aMem[i]);
-			p->aMem[i].flags = MEM_Undefined;
+			mem_set_undefined(&p->aMem[i]);
 		}
 		memset(p->apCsr, 0, nCursor * sizeof(VdbeCursor *));
 	}
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 36/53] sql: introduce mem_set_pointer()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (34 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 35/53] sql: introduce mem_set_undefined() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 37/53] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
                   ` (16 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_pointer() function. Function
mem_set_pointer() sets a pointer to MEM. This pointer used for internal
VDBE operations.

Part of #5818
---
 src/box/sql/mem.c     | 17 +++++++++--------
 src/box/sql/mem.h     | 11 +++--------
 src/box/sql/vdbe.c    |  3 +--
 src/box/sql/vdbeapi.c |  2 +-
 4 files changed, 14 insertions(+), 19 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index fd2710478..3701741ef 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -595,6 +595,15 @@ mem_set_undefined(struct Mem *mem)
 	mem->field_type = field_type_MAX;
 }
 
+void
+mem_set_pointer(struct Mem *mem, void *ptr)
+{
+	mem_clear(mem);
+	mem->flags = MEM_Ptr;
+	mem->u.p = ptr;
+	mem->field_type = field_type_MAX;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2213,14 +2222,6 @@ sqlVdbeMemClearAndResize(Mem * pMem, int szNew)
 	return 0;
 }
 
-void
-mem_set_ptr(struct Mem *mem, void *ptr)
-{
-	mem_destroy(mem);
-	mem->flags = MEM_Ptr;
-	mem->u.p = ptr;
-}
-
 /*
  * Free an sql_value object
  */
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 75486ba3a..9fec6ed8a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -265,6 +265,9 @@ mem_set_allocated_array(struct Mem *mem, char *value, uint32_t size);
 void
 mem_set_undefined(struct Mem *mem);
 
+void
+mem_set_pointer(struct Mem *mem, void *ptr);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
@@ -492,14 +495,6 @@ mem_convert_to_numeric(struct Mem *mem, enum field_type type);
 int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
 int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
 
-/**
- * Set VDBE memory register with given pointer as a data.
- * @param mem VDBE memory register to update.
- * @param ptr Pointer to use.
- */
-void
-mem_set_ptr(struct Mem *mem, void *ptr);
-
 void sqlValueFree(struct Mem *);
 struct Mem *sqlValueNew(struct sql *);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index f899dfe7a..b9c2142e8 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2394,8 +2394,7 @@ case OP_OpenTEphemeral: {
 
 	if (space == NULL)
 		goto abort_due_to_error;
-	aMem[pOp->p1].u.p = space;
-	aMem[pOp->p1].flags = MEM_Ptr;
+	mem_set_pointer(&aMem[pOp->p1], space);
 	break;
 }
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index d6d6e54f5..3ad89b87c 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -930,7 +930,7 @@ sql_bind_ptr(struct sql_stmt *stmt, int i, void *ptr)
 	int rc = vdbeUnbind(p, i);
 	if (rc == 0) {
 		rc = sql_bind_type(p, i, "varbinary");
-		mem_set_ptr(&p->aVar[i - 1], ptr);
+		mem_set_pointer(&p->aVar[i - 1], ptr);
 	}
 	return rc;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 37/53] sql: introduce mem_set_frame()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (35 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 36/53] sql: introduce mem_set_pointer() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 38/53] sql: introduce mem_*_aggregate() Mergen Imeev via Tarantool-patches
                   ` (15 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_frame() function. Function
mem_set_frame() sets a frame to MEM. This frame used for internal
VDBE operations.

Part of #5818
---
 src/box/sql/mem.c  | 9 +++++++++
 src/box/sql/mem.h  | 3 +++
 src/box/sql/vdbe.c | 3 +--
 3 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 3701741ef..078de0e62 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -604,6 +604,15 @@ mem_set_pointer(struct Mem *mem, void *ptr)
 	mem->field_type = field_type_MAX;
 }
 
+void
+mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
+{
+	mem_clear(mem);
+	mem->flags = MEM_Frame;
+	mem->u.pFrame = frame;
+	mem->field_type = field_type_MAX;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 9fec6ed8a..a6c53c615 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -268,6 +268,9 @@ mem_set_undefined(struct Mem *mem);
 void
 mem_set_pointer(struct Mem *mem, void *ptr);
 
+void
+mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index b9c2142e8..b01b8d760 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -4073,8 +4073,7 @@ case OP_Program: {        /* jump */
 			goto no_mem;
 		}
 		mem_destroy(pRt);
-		pRt->flags = MEM_Frame;
-		pRt->u.pFrame = pFrame;
+		mem_set_frame(pRt, pFrame);
 
 		pFrame->v = p;
 		pFrame->nChildMem = nMem;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 38/53] sql: introduce mem_*_aggregate()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (36 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 37/53] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 39/53] sql: introduce mem_set_cleared() Mergen Imeev via Tarantool-patches
                   ` (14 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduce mem_prepare_aggregare() and mem_get_aggregate() functions.
These functions are used during execution of aggregate functions.

Part of #5818
---
 src/box/sql/mem.c     | 23 +++++++++++++++++++++++
 src/box/sql/mem.h     |  6 ++++++
 src/box/sql/vdbeapi.c | 33 ++++-----------------------------
 3 files changed, 33 insertions(+), 29 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 078de0e62..0211069c6 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -613,6 +613,29 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
 	mem->field_type = field_type_MAX;
 }
 
+int
+mem_prepare_aggregate(struct Mem *mem, struct func *func, int size)
+{
+	if (size <= 0) {
+		mem_clear(mem);
+		return 0;
+	}
+	if (sqlVdbeMemGrow(mem, size, 0) != 0)
+		return -1;
+	memset(mem->z, 0, size);
+	mem->n = size;
+	mem->flags = MEM_Agg;
+	mem->u.func = func;
+	mem->field_type = field_type_MAX;
+	return 0;
+}
+
+void *
+mem_get_aggregate(struct Mem *mem)
+{
+	return (void *)mem->z;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index a6c53c615..51802b450 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -271,6 +271,12 @@ mem_set_pointer(struct Mem *mem, void *ptr);
 void
 mem_set_frame(struct Mem *mem, struct VdbeFrame *frame);
 
+int
+mem_prepare_aggregate(struct Mem *mem, struct func *func, int size);
+
+void *
+mem_get_aggregate(struct Mem *mem);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 3ad89b87c..8f871ea95 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -403,29 +403,6 @@ sqlStmtCurrentTime(sql_context * p)
 	return *piTime;
 }
 
-/*
- * Create a new aggregate context for p and return a pointer to
- * its pMem->z element.
- */
-static SQL_NOINLINE void *
-createAggContext(sql_context * p, int nByte)
-{
-	Mem *pMem = p->pMem;
-	assert(!mem_is_aggregate(pMem));
-	if (nByte <= 0) {
-		mem_set_null(pMem);
-		pMem->z = 0;
-	} else {
-		sqlVdbeMemClearAndResize(pMem, nByte);
-		pMem->flags = MEM_Agg;
-		pMem->u.func = p->func;
-		if (pMem->z) {
-			memset(pMem->z, 0, nByte);
-		}
-	}
-	return (void *)pMem->z;
-}
-
 /*
  * Allocate or return the aggregate context for a user function.  A new
  * context is allocated on the first call.  Subsequent calls return the
@@ -437,12 +414,10 @@ sql_aggregate_context(sql_context * p, int nByte)
 	assert(p != NULL && p->func != NULL);
 	assert(p->func->def->language == FUNC_LANGUAGE_SQL_BUILTIN);
 	assert(p->func->def->aggregate == FUNC_AGGREGATE_GROUP);
-	testcase(nByte < 0);
-	if (!mem_is_aggregate(p->pMem)) {
-		return createAggContext(p, nByte);
-	} else {
-		return (void *)p->pMem->z;
-	}
+	if (!mem_is_aggregate(p->pMem) &&
+	    mem_prepare_aggregate(p->pMem, p->func, nByte) != 0)
+		return NULL;
+	return mem_get_aggregate(p->pMem);
 }
 
 /*
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 39/53] sql: introduce mem_set_cleared()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (37 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 38/53] sql: introduce mem_*_aggregate() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 40/53] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
                   ` (13 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_set_cleared() function. Function
mem_set_cleared() sets a NULL to MEM, however this NULL should be used
only in internal VDBE operations.

Part of #5818
---
 src/box/sql/mem.c  | 7 +++++++
 src/box/sql/mem.h  | 3 +++
 src/box/sql/vdbe.c | 9 +++++----
 3 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 0211069c6..83f17250d 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -636,6 +636,13 @@ mem_get_aggregate(struct Mem *mem)
 	return (void *)mem->z;
 }
 
+void
+mem_set_cleared(struct Mem *mem)
+{
+	mem_clear(mem);
+	mem->flags = MEM_Null | MEM_Cleared;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 51802b450..70224b55a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -277,6 +277,9 @@ mem_prepare_aggregate(struct Mem *mem, struct func *func, int size);
 void *
 mem_get_aggregate(struct Mem *mem);
 
+void
+mem_set_cleared(struct Mem *mem);
+
 int
 mem_copy(struct Mem *to, const struct Mem *from);
 
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index b01b8d760..fdcb4d1e6 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -823,7 +823,7 @@ case OP_String: {          /* out2 */
  * is less than P2 (typically P3 is zero) then only register P2 is
  * set to NULL.
  *
- * If the P1 value is non-zero, then also set the MEM_Cleared flag so that
+ * If the P1 value is non-zero, then also set the Cleared flag so that
  * NULL values will not compare equal even if SQL_NULLEQ is set on
  * OP_Ne or OP_Eq.
  */
@@ -833,13 +833,14 @@ case OP_Null: {           /* out2 */
 	cnt = pOp->p3-pOp->p2;
 	assert(pOp->p3<=(p->nMem+1 - p->nCursor));
 	if (pOp->p1 != 0)
-		pOut->flags = MEM_Null|MEM_Cleared;
+		mem_set_cleared(pOut);
 	while( cnt>0) {
 		pOut++;
 		memAboutToChange(p, pOut);
-		mem_set_null(pOut);
 		if (pOp->p1 != 0)
-			pOut->flags = MEM_Null|MEM_Cleared;
+			mem_set_cleared(pOut);
+		else
+			mem_set_null(pOut);
 		cnt--;
 	}
 	break;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 40/53] sql: move MEM flags to mem.c
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (38 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 39/53] sql: introduce mem_set_cleared() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 41/53] sql: introduce mem_convert_to_integer() Mergen Imeev via Tarantool-patches
                   ` (12 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch moves MEM flags to mem.c. This allow us to have more control
over MEM state.

Part of #5818
---
 src/box/sql/mem.c  | 52 ++++++++++++++++++++++++++++++++++++++
 src/box/sql/mem.h  | 63 ++--------------------------------------------
 src/box/sql/vdbe.c |  3 ---
 3 files changed, 54 insertions(+), 64 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 83f17250d..1209df1ce 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -37,6 +37,58 @@
 #include "box/tuple.h"
 #include "mpstream/mpstream.h"
 
+/* One or more of the following flags are set to indicate the validOK
+ * representations of the value stored in the Mem struct.
+ *
+ * If the MEM_Null flag is set, then the value is an SQL NULL value.
+ * No other flags may be set in this case.
+ *
+ * If the MEM_Str flag is set then Mem.z points at a string representation.
+ * Usually this is encoded in the same unicode encoding as the main
+ * database (see below for exceptions). If the MEM_Term flag is also
+ * set, then the string is nul terminated. The MEM_Int and MEM_Real
+ * flags may coexist with the MEM_Str flag.
+ */
+#define MEM_Null      0x0001	/* Value is NULL */
+#define MEM_Str       0x0002	/* Value is a string */
+#define MEM_Int       0x0004	/* Value is an integer */
+#define MEM_Real      0x0008	/* Value is a real number */
+#define MEM_Blob      0x0010	/* Value is a BLOB */
+#define MEM_Bool      0x0020    /* Value is a bool */
+#define MEM_UInt      0x0040	/* Value is an unsigned integer */
+#define MEM_Frame     0x0080	/* Value is a VdbeFrame object */
+#define MEM_Undefined 0x0100	/* Value is undefined */
+#define MEM_Cleared   0x0200	/* NULL set by OP_Null, not from data */
+#define MEM_TypeMask  0x83ff	/* Mask of type bits */
+
+/* Whenever Mem contains a valid string or blob representation, one of
+ * the following flags must be set to determine the memory management
+ * policy for Mem.z.  The MEM_Term flag tells us whether or not the
+ * string is \000 or \u0000 terminated
+ */
+#define MEM_Term      0x0400	/* String rep is nul terminated */
+#define MEM_Dyn       0x0800	/* Need to call Mem.xDel() on Mem.z */
+#define MEM_Static    0x1000	/* Mem.z points to a static string */
+#define MEM_Ephem     0x2000	/* Mem.z points to an ephemeral string */
+#define MEM_Agg       0x4000	/* Mem.z points to an agg function context */
+#define MEM_Zero      0x8000	/* Mem.i contains count of 0s appended to blob */
+#define MEM_Subtype   0x10000	/* Mem.eSubtype is valid */
+#define MEM_Ptr       0x20000	/* Value is a generic pointer */
+
+/**
+ * In contrast to Mem_TypeMask, this one allows to get
+ * pure type of memory cell, i.e. without _Dyn/_Zero and other
+ * auxiliary flags.
+ */
+enum {
+	MEM_PURE_TYPE_MASK = 0x7f
+};
+
+static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
+				     MEM_Blob | MEM_Bool | MEM_UInt),
+	      "value of type mask must consist of corresponding to memory "\
+	      "type bits");
+
 bool
 mem_is_null(const struct Mem *mem)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 70224b55a..8b6f6749d 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -318,58 +318,6 @@ int
 mem_compare(const struct Mem *left, const struct Mem *right, int *result,
 	    enum field_type type, struct coll *coll);
 
-/* One or more of the following flags are set to indicate the validOK
- * representations of the value stored in the Mem struct.
- *
- * If the MEM_Null flag is set, then the value is an SQL NULL value.
- * No other flags may be set in this case.
- *
- * If the MEM_Str flag is set then Mem.z points at a string representation.
- * Usually this is encoded in the same unicode encoding as the main
- * database (see below for exceptions). If the MEM_Term flag is also
- * set, then the string is nul terminated. The MEM_Int and MEM_Real
- * flags may coexist with the MEM_Str flag.
- */
-#define MEM_Null      0x0001	/* Value is NULL */
-#define MEM_Str       0x0002	/* Value is a string */
-#define MEM_Int       0x0004	/* Value is an integer */
-#define MEM_Real      0x0008	/* Value is a real number */
-#define MEM_Blob      0x0010	/* Value is a BLOB */
-#define MEM_Bool      0x0020    /* Value is a bool */
-#define MEM_UInt      0x0040	/* Value is an unsigned integer */
-#define MEM_Frame     0x0080	/* Value is a VdbeFrame object */
-#define MEM_Undefined 0x0100	/* Value is undefined */
-#define MEM_Cleared   0x0200	/* NULL set by OP_Null, not from data */
-#define MEM_TypeMask  0x83ff	/* Mask of type bits */
-
-/* Whenever Mem contains a valid string or blob representation, one of
- * the following flags must be set to determine the memory management
- * policy for Mem.z.  The MEM_Term flag tells us whether or not the
- * string is \000 or \u0000 terminated
- */
-#define MEM_Term      0x0400	/* String rep is nul terminated */
-#define MEM_Dyn       0x0800	/* Need to call Mem.xDel() on Mem.z */
-#define MEM_Static    0x1000	/* Mem.z points to a static string */
-#define MEM_Ephem     0x2000	/* Mem.z points to an ephemeral string */
-#define MEM_Agg       0x4000	/* Mem.z points to an agg function context */
-#define MEM_Zero      0x8000	/* Mem.i contains count of 0s appended to blob */
-#define MEM_Subtype   0x10000	/* Mem.eSubtype is valid */
-#define MEM_Ptr       0x20000	/* Value is a generic pointer */
-
-/**
- * In contrast to Mem_TypeMask, this one allows to get
- * pure type of memory cell, i.e. without _Dyn/_Zero and other
- * auxiliary flags.
- */
-enum {
-	MEM_PURE_TYPE_MASK = 0x7f
-};
-
-static_assert(MEM_PURE_TYPE_MASK == (MEM_Null | MEM_Str | MEM_Int | MEM_Real |
-				     MEM_Blob | MEM_Bool | MEM_UInt),
-	      "value of type mask must consist of corresponding to memory "\
-	      "type bits");
-
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -402,7 +350,7 @@ registerTrace(int iReg, Mem *p);
  * Return true if a memory cell is not marked as invalid.  This macro
  * is for use inside assert() statements only.
  */
-#define memIsValid(M)  ((M)->flags & MEM_Undefined)==0
+#define memIsValid(M) !mem_is_undefined(M)
 #endif
 
 /*
@@ -451,7 +399,7 @@ int mem_apply_integer_type(struct Mem *);
 int sqlVdbeMemStringify(struct Mem *);
 int sqlVdbeMemNulTerminate(struct Mem *);
 int sqlVdbeMemExpandBlob(struct Mem *);
-#define ExpandBlob(P) (((P)->flags&MEM_Zero)?sqlVdbeMemExpandBlob(P):0)
+#define ExpandBlob(P) (mem_is_zeroblob(P)? sqlVdbeMemExpandBlob(P) : 0)
 void sql_value_apply_type(struct Mem *val, enum field_type type);
 
 
@@ -563,13 +511,6 @@ columnNullValue(void);
 
 int sqlVdbeMemTooBig(Mem *);
 
-/* Return TRUE if Mem X contains dynamically allocated content - anything
- * that needs to be deallocated to avoid a leak.
- */
-#define VdbeMemDynamic(X)  \
-  (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)
-
-
 int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
 
 /**
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index fdcb4d1e6..edf845d97 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -575,7 +575,6 @@ case OP_SetDiag: {             /* jump */
 case OP_Gosub: {            /* jump */
 	assert(pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor));
 	pIn1 = &aMem[pOp->p1];
-	assert(VdbeMemDynamic(pIn1)==0);
 	memAboutToChange(p, pIn1);
 	mem_set_unsigned(pIn1, pOp - aOp);
 	REGISTER_TRACE(p, pOp->p1, pIn1);
@@ -617,7 +616,6 @@ case OP_InitCoroutine: {     /* jump */
 	assert(pOp->p2>=0 && pOp->p2<p->nOp);
 	assert(pOp->p3>0 && pOp->p3<p->nOp);
 	pOut = &aMem[pOp->p1];
-	assert(!VdbeMemDynamic(pOut));
 	mem_set_unsigned(pOut, pOp->p3 - 1);
 	if (pOp->p2) goto jump_to_p2;
 	break;
@@ -659,7 +657,6 @@ case OP_EndCoroutine: {           /* in1 */
  */
 case OP_Yield: {            /* in1, jump */
 	pIn1 = &aMem[pOp->p1];
-	assert(VdbeMemDynamic(pIn1)==0);
 	int pcDest = (int)pIn1->u.u;
 	mem_set_unsigned(pIn1, pOp - aOp);
 	REGISTER_TRACE(p, pOp->p1, pIn1);
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 41/53] sql: introduce mem_convert_to_integer()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (39 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 40/53] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 42/53] sql: introduce mem_convert_to_double() Mergen Imeev via Tarantool-patches
                   ` (11 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_convert_to_integer() which is used to convert
a MEM to a MEM that contains integer value.

Part of #5818
---
 src/box/sql/mem.c  | 114 +++++++++++++++++++++++++++++----------------
 src/box/sql/mem.h  |   7 ++-
 src/box/sql/vdbe.c |  19 ++++----
 3 files changed, 88 insertions(+), 52 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 1209df1ce..b9bcd3d3a 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -695,6 +695,80 @@ mem_set_cleared(struct Mem *mem)
 	mem->flags = MEM_Null | MEM_Cleared;
 }
 
+static inline int
+mem_convert_varstring_to_integer(struct Mem *mem)
+{
+	bool is_neg;
+	int64_t i;
+	if (sql_atoi64(mem->z, &i, &is_neg, mem->n) != 0)
+		return -1;
+	mem_set_integer(mem, i, is_neg);
+	return 0;
+}
+
+static inline int
+mem_convert_double_to_integer(struct Mem *mem)
+{
+	double d = mem->u.r;
+	if (d < 0 && d >= (double)INT64_MIN) {
+		mem_set_integer(mem, (int64_t)d, true);
+		return 0;
+	}
+	if (d >= 0 && d < (double)UINT64_MAX) {
+		mem_set_integer(mem, (int64_t)(uint64_t)d, false);
+		return 0;
+	}
+	return -1;
+}
+
+static inline int
+mem_convert_double_to_integer_lossless(struct Mem *mem)
+{
+	double d = mem->u.r;
+	if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
+		mem_set_integer(mem, (int64_t)d, true);
+		return 0;
+	}
+	if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
+		mem_set_integer(mem, (int64_t)(uint64_t)d, false);
+		return 0;
+	}
+	return -1;
+}
+
+static inline int
+mem_convert_boolean_to_integer(struct Mem *mem)
+{
+	mem_set_integer(mem, (int64_t)mem->u.b, false);
+	return 0;
+}
+
+int
+mem_convert_to_integer(struct Mem *mem)
+{
+	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+		return 0;
+	if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+		return mem_convert_varstring_to_integer(mem);
+	if ((mem->flags & MEM_Real) != 0)
+		return mem_convert_double_to_integer(mem);
+	if ((mem->flags & MEM_Bool) != 0)
+		return mem_convert_boolean_to_integer(mem);
+	return -1;
+}
+
+int
+mem_convert_to_integer_lossless(struct Mem *mem)
+{
+	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+		return 0;
+	if ((mem->flags & MEM_Str) != 0)
+		return mem_convert_varstring_to_integer(mem);
+	if ((mem->flags & MEM_Real) != 0)
+		return mem_convert_double_to_integer_lossless(mem);
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1911,23 +1985,6 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 	}
 }
 
-/*
- * The MEM structure is already a MEM_Real.  Try to also make it a
- * MEM_Int if we can.
- */
-int
-mem_apply_integer_type(Mem *pMem)
-{
-	int rc;
-	i64 ix;
-	assert(pMem->flags & MEM_Real);
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
-	if ((rc = doubleToInt64(pMem->u.r, (int64_t *) &ix)) == 0)
-		mem_set_integer(pMem, ix, pMem->u.r <= -1);
-	return rc;
-}
-
 /*
  * Add MEM_Str to the set of representations for the given Mem.  Numbers
  * are converted using sql_snprintf().  Converting a BLOB to a string
@@ -2192,29 +2249,6 @@ mem_convert_to_unsigned(struct Mem *mem)
 	return 0;
 }
 
-/**
- * Convert the numeric value contained in MEM to integer.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_integer(struct Mem *mem)
-{
-	if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
-		return 0;
-	if ((mem->flags & MEM_Real) == 0)
-		return -1;
-	double d = mem->u.r;
-	if (d >= (double)UINT64_MAX || d < (double)INT64_MIN)
-		return -1;
-	if (d < (double)INT64_MAX)
-		mem_set_integer(mem, (int64_t) d, d < 0);
-	else
-		mem_set_integer(mem, (uint64_t) d, false);
-	return 0;
-}
-
 int
 mem_convert_to_numeric(struct Mem *mem, enum field_type type)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8b6f6749d..58cbdfe9f 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -318,6 +318,12 @@ int
 mem_compare(const struct Mem *left, const struct Mem *right, int *result,
 	    enum field_type type, struct coll *coll);
 
+int
+mem_convert_to_integer(struct Mem *mem);
+
+int
+mem_convert_to_integer_lossless(struct Mem *mem);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -395,7 +401,6 @@ int
 vdbe_mem_numerify(struct Mem *mem);
 
 int sqlVdbeMemCast(struct Mem *, enum field_type type);
-int mem_apply_integer_type(struct Mem *);
 int sqlVdbeMemStringify(struct Mem *);
 int sqlVdbeMemNulTerminate(struct Mem *);
 int sqlVdbeMemExpandBlob(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index edf845d97..1aeed543f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1341,17 +1341,14 @@ case OP_AddImm: {            /* in1 */
  */
 case OP_MustBeInt: {            /* jump, in1 */
 	pIn1 = &aMem[pOp->p1];
-	if (!mem_is_integer(pIn1)) {
-		mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
-		if (!mem_is_integer(pIn1)) {
-			if (pOp->p2==0) {
-				diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-					 mem_str(pIn1), "integer");
-				goto abort_due_to_error;
-			} else {
-				goto jump_to_p2;
-			}
-		}
+	if (mem_is_integer(pIn1))
+		break;
+	if (mem_convert_to_integer_lossless(pIn1) != 0) {
+		if (pOp->p2 !=0 )
+			goto jump_to_p2;
+		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+			 mem_str(pIn1), "integer");
+		goto abort_due_to_error;
 	}
 	break;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 42/53] sql: introduce mem_convert_to_double()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (40 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 41/53] sql: introduce mem_convert_to_integer() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 43/53] sql: introduce mem_convert_to_number() Mergen Imeev via Tarantool-patches
                   ` (10 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch intruduces mem_convert_to_double(). Function
mem_convert_to_double() is used to convert MEM to MEM contains double
values.

Part of #5818
---
 src/box/sql/mem.c  | 77 ++++++++++++++++++++++------------------------
 src/box/sql/mem.h  |  4 ++-
 src/box/sql/vdbe.c |  2 +-
 3 files changed, 41 insertions(+), 42 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index b9bcd3d3a..8f2c48cd5 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -769,6 +769,40 @@ mem_convert_to_integer_lossless(struct Mem *mem)
 	return -1;
 }
 
+static inline int
+mem_convert_integer_to_double(struct Mem *mem)
+{
+	double d;
+	if ((mem->flags & MEM_UInt) != 0)
+		d = (double)mem->u.u;
+	else
+		d = (double)mem->u.i;
+	mem_set_double(mem, d);
+	return 0;
+}
+
+static inline int
+mem_convert_varstring_to_double(struct Mem *mem)
+{
+	double d;
+	if (sqlAtoF(mem->z, &d, mem->n) == 0)
+		return -1;
+	mem_set_double(mem, d);
+	return 0;
+}
+
+int
+mem_convert_to_double(struct Mem *mem)
+{
+	if ((mem->flags & MEM_Real) != 0)
+		return 0;
+	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+		return mem_convert_integer_to_double(mem);
+	if ((mem->flags & MEM_Str) != 0)
+		return mem_convert_varstring_to_double(mem);
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1844,21 +1878,6 @@ mem_apply_numeric_type(struct Mem *record)
 	return 0;
 }
 
-/*
- * Convert pMem so that it is of type MEM_Real.
- * Invalidate any prior representations.
- */
-int
-sqlVdbeMemRealify(Mem * pMem)
-{
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-	double v;
-	if (sqlVdbeRealValue(pMem, &v))
-		return -1;
-	mem_set_double(pMem, v);
-	return 0;
-}
-
 int
 vdbe_mem_numerify(struct Mem *mem)
 {
@@ -1956,7 +1975,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 			return -1;
 		return 0;
 	case FIELD_TYPE_DOUBLE:
-		return sqlVdbeMemRealify(pMem);
+		return mem_convert_to_double(pMem);
 	case FIELD_TYPE_NUMBER:
 		return vdbe_mem_numerify(pMem);
 	case FIELD_TYPE_VARBINARY:
@@ -2157,11 +2176,11 @@ mem_apply_type(struct Mem *record, enum field_type type)
 	case FIELD_TYPE_NUMBER:
 		if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
 			return 0;
-		return sqlVdbeMemRealify(record);
+		return mem_convert_to_double(record);
 	case FIELD_TYPE_DOUBLE:
 		if ((record->flags & MEM_Real) != 0)
 			return 0;
-		return sqlVdbeMemRealify(record);
+		return mem_convert_to_double(record);
 	case FIELD_TYPE_STRING:
 		/*
 		 * Only attempt the conversion to TEXT if there is
@@ -2205,28 +2224,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
 	}
 }
 
-/**
- * Convert the numeric value contained in MEM to double.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_double(struct Mem *mem)
-{
-	if ((mem->flags & MEM_Real) != 0)
-		return 0;
-	if ((mem->flags & (MEM_Int | MEM_UInt)) == 0)
-		return -1;
-	double d;
-	if ((mem->flags & MEM_Int) != 0)
-		d = (double)mem->u.i;
-	else
-		d = (double)mem->u.u;
-	mem_set_double(mem, d);
-	return 0;
-}
-
 /**
  * Convert the numeric value contained in MEM to unsigned.
  *
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 58cbdfe9f..be793d08a 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -324,6 +324,9 @@ mem_convert_to_integer(struct Mem *mem);
 int
 mem_convert_to_integer_lossless(struct Mem *mem);
 
+int
+mem_convert_to_double(struct Mem *mem);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -387,7 +390,6 @@ registerTrace(int iReg, Mem *p);
  */
 int
 mem_apply_numeric_type(struct Mem *record);
-int sqlVdbeMemRealify(struct Mem *);
 
 /**
  * Convert @a mem to NUMBER type, so that after conversion it has
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 1aeed543f..3c02eae74 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1365,7 +1365,7 @@ case OP_MustBeInt: {            /* jump, in1 */
 case OP_Realify: {                  /* in1 */
 	pIn1 = &aMem[pOp->p1];
 	if (mem_is_integer(pIn1)) {
-		sqlVdbeMemRealify(pIn1);
+		mem_convert_to_double(pIn1);
 	}
 	break;
 }
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 43/53] sql: introduce mem_convert_to_number()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (41 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 42/53] sql: introduce mem_convert_to_double() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 44/53] sql: introduce mem_convert_to_string() Mergen Imeev via Tarantool-patches
                   ` (9 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_convert_to_number(). This function is used to
convert MEM to MEM contains number.

Part of #5818
---
 src/box/sql/func.c |  2 +-
 src/box/sql/mem.c  | 58 +++++++++++++---------------------------------
 src/box/sql/mem.h  | 28 +++-------------------
 src/box/sql/vdbe.c |  2 +-
 4 files changed, 21 insertions(+), 69 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 1b3929259..b644c39d8 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1948,7 +1948,7 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
 	if (type == MP_NIL || p == NULL)
 		return;
 	if (type != MP_DOUBLE && type != MP_INT && type != MP_UINT) {
-		if (mem_apply_numeric_type(argv[0]) != 0) {
+		if (type != MP_STR || mem_convert_to_number(argv[0]) != 0) {
 			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 				 mem_str(argv[0]), "number");
 			context->is_aborted = true;
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 8f2c48cd5..8600b5c41 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -803,6 +803,21 @@ mem_convert_to_double(struct Mem *mem)
 	return -1;
 }
 
+int
+mem_convert_to_number(struct Mem *mem)
+{
+	if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+		return 0;
+	if ((mem->flags & MEM_Bool) != 0)
+		return mem_convert_boolean_to_integer(mem);
+	if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) {
+		if (mem_convert_varstring_to_integer(mem) == 0)
+			return 0;
+		return mem_convert_varstring_to_double(mem);
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1860,47 +1875,6 @@ registerTrace(int iReg, Mem *p) {
 }
 #endif
 
-int
-mem_apply_numeric_type(struct Mem *record)
-{
-	if ((record->flags & MEM_Str) == 0)
-		return -1;
-	int64_t integer_value;
-	bool is_neg;
-	if (sql_atoi64(record->z, &integer_value, &is_neg, record->n) == 0) {
-		mem_set_integer(record, integer_value, is_neg);
-		return 0;
-	}
-	double float_value;
-	if (sqlAtoF(record->z, &float_value, record->n) == 0)
-		return -1;
-	mem_set_double(record, float_value);
-	return 0;
-}
-
-int
-vdbe_mem_numerify(struct Mem *mem)
-{
-	if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
-		return 0;
-	if ((mem->flags & MEM_Bool) != 0) {
-		mem_set_integer(mem, (int64_t)mem->u.b, false);
-		return 0;
-	}
-	assert((mem->flags & (MEM_Blob | MEM_Str)) != 0);
-	bool is_neg;
-	int64_t i;
-	if (sql_atoi64(mem->z, &i, &is_neg, mem->n) == 0) {
-		mem_set_integer(mem, i, is_neg);
-	} else {
-		double d;
-		if (sqlAtoF(mem->z, &d, mem->n) == 0)
-			return -1;
-		mem_set_double(mem, d);
-	}
-	return 0;
-}
-
 /*
  * Cast the datatype of the value in pMem according to the type
  * @type.  Casting is different from applying type in that a cast
@@ -1977,7 +1951,7 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 	case FIELD_TYPE_DOUBLE:
 		return mem_convert_to_double(pMem);
 	case FIELD_TYPE_NUMBER:
-		return vdbe_mem_numerify(pMem);
+		return mem_convert_to_number(pMem);
 	case FIELD_TYPE_VARBINARY:
 		if ((pMem->flags & MEM_Blob) != 0)
 			return 0;
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index be793d08a..d5f0d7891 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -327,6 +327,9 @@ mem_convert_to_integer_lossless(struct Mem *mem);
 int
 mem_convert_to_double(struct Mem *mem);
 
+int
+mem_convert_to_number(struct Mem *mem);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -377,31 +380,6 @@ registerTrace(int iReg, Mem *p);
 # define memAboutToChange(P,M)
 #endif
 
-/**
- * Try to convert a string value into a numeric representation
- * if we can do so without loss of information. Firstly, value
- * is attempted to be converted to integer, and in case of fail -
- * to floating point number. Note that function is assumed to be
- * called on memory cell containing string, i.e. mem->type == MEM_Str.
- *
- * @param record Memory cell containing value to be converted.
- * @retval 0 If value can be converted to integer or number.
- * @retval -1 Otherwise.
- */
-int
-mem_apply_numeric_type(struct Mem *record);
-
-/**
- * Convert @a mem to NUMBER type, so that after conversion it has
- * one of types MEM_Real, MEM_Int or MEM_UInt. If conversion is
- * not possible, function returns -1.
- *
- * Beware - this function changes value and type of @a mem
- * argument.
- */
-int
-vdbe_mem_numerify(struct Mem *mem);
-
 int sqlVdbeMemCast(struct Mem *, enum field_type type);
 int sqlVdbeMemStringify(struct Mem *);
 int sqlVdbeMemNulTerminate(struct Mem *);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 3c02eae74..6799cc9aa 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2621,7 +2621,7 @@ case OP_SeekGT: {       /* jump, in3 */
 		if (mem_is_null(pIn3))
 			goto skip_truncate;
 		if (mem_is_string(pIn3))
-			mem_apply_numeric_type(pIn3);
+			mem_convert_to_number(pIn3);
 		int64_t i;
 		if (mem_is_integer(pIn3) && !mem_is_unsigned(pIn3)) {
 			i = pIn3->u.i;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 44/53] sql: introduce mem_convert_to_string()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (42 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 43/53] sql: introduce mem_convert_to_number() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 45/53] sql: introduce mem_explicit_cast() Mergen Imeev via Tarantool-patches
                   ` (8 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_convert_to_string(). This function is used to
convert MEM to MEM contains string value.

Part of #5818
---
 src/box/sql/mem.c | 197 ++++++++++++++++++++++++++++++----------------
 src/box/sql/mem.h |   7 +-
 2 files changed, 133 insertions(+), 71 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 8600b5c41..262f48aca 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -818,6 +818,131 @@ mem_convert_to_number(struct Mem *mem)
 	return -1;
 }
 
+static inline int
+mem_convert_integer_to_string(struct Mem *mem)
+{
+	const char *str;
+	if ((mem->flags & MEM_UInt) != 0)
+		str = tt_sprintf("%llu", mem->u.u);
+	else
+		str = tt_sprintf("%lld", mem->u.i);
+	return mem_copy_string0(mem, str);
+}
+
+static inline int
+mem_convert_array_to_string(struct Mem *mem)
+{
+	const char *str = mp_str(mem->z);
+	return mem_copy_string0(mem, str);
+}
+
+static inline int
+mem_convert_map_to_string(struct Mem *mem)
+{
+	const char *str = mp_str(mem->z);
+	return mem_copy_string0(mem, str);
+}
+
+static inline int
+mem_convert_boolean_to_string(struct Mem *mem)
+{
+	const char *str = mem->u.b ? "TRUE" : "FALSE";
+	return mem_copy_string0(mem, str);
+}
+
+static inline int
+mem_convert_binary_to_string(struct Mem *mem)
+{
+	if (ExpandBlob(mem) != 0)
+		return -1;
+	mem->flags = (mem->flags & (MEM_Dyn | MEM_Static | MEM_Ephem)) |
+		      MEM_Str;
+	mem->field_type = FIELD_TYPE_STRING;
+	return 0;
+}
+
+static inline int
+mem_convert_binary_to_string0(struct Mem *mem)
+{
+	if (ExpandBlob(mem) != 0)
+		return -1;
+	if (sqlVdbeMemGrow(mem, mem->n + 1, 1) != 0)
+		return -1;
+	mem->z[mem->n] = '\0';
+	mem->flags = MEM_Str | MEM_Term;
+	mem->field_type = FIELD_TYPE_STRING;
+	return 0;
+}
+
+static inline int
+mem_convert_double_to_string(struct Mem *mem)
+{
+	uint32_t size = 32;
+	if (sqlVdbeMemClearAndResize(mem, size) != 0)
+		return -1;
+	sql_snprintf(size, mem->z, "%!.15g", mem->u.r);
+	mem->n = strlen(mem->z);
+	mem->flags = MEM_Str | MEM_Term;
+	mem->field_type = FIELD_TYPE_STRING;
+	return 0;
+}
+
+static inline int
+mem_convert_string_to_string0(struct Mem *mem)
+{
+	assert((mem->flags | MEM_Str) != 0);
+	if (sqlVdbeMemGrow(mem, mem->n + 1, 1) != 0)
+		return -1;
+	mem->z[mem->n] = '\0';
+	mem->flags |= MEM_Term;
+	mem->field_type = FIELD_TYPE_STRING;
+	return 0;
+}
+
+int
+mem_convert_to_string0(struct Mem *mem)
+{
+	if ((mem->flags & MEM_Str) != 0 && (mem->flags & MEM_Term) != 0)
+		return 0;
+	if ((mem->flags & MEM_Str) != 0)
+		return mem_convert_string_to_string0(mem);
+	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+		return mem_convert_integer_to_string(mem);
+	if ((mem->flags & MEM_Real) != 0)
+		return mem_convert_double_to_string(mem);
+	if ((mem->flags & MEM_Bool) != 0)
+		return mem_convert_boolean_to_string(mem);
+	if ((mem->flags & MEM_Blob) != 0) {
+		if ((mem->flags & MEM_Subtype) == 0)
+			return mem_convert_binary_to_string0(mem);
+		if (mp_typeof(*mem->z) == MP_MAP)
+			return mem_convert_map_to_string(mem);
+		return mem_convert_array_to_string(mem);
+	}
+	return -1;
+}
+
+int
+mem_convert_to_string(struct Mem *mem)
+{
+	if ((mem->flags & MEM_Str) != 0)
+		return 0;
+	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+		return mem_convert_integer_to_string(mem);
+	if ((mem->flags & MEM_Real) != 0)
+		return mem_convert_double_to_string(mem);
+	if ((mem->flags & MEM_Bool) != 0)
+		return mem_convert_boolean_to_string(mem);
+	if ((mem->flags & MEM_Blob) != 0) {
+		if ((mem->flags & MEM_Subtype) == 0)
+			return mem_convert_binary_to_string(mem);
+		if (mp_typeof(*mem->z) == MP_MAP)
+			return mem_convert_map_to_string(mem);
+		return mem_convert_array_to_string(mem);
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1452,7 +1577,7 @@ valueToText(sql_value * pVal)
 		pVal->flags |= MEM_Str;
 		sqlVdbeMemNulTerminate(pVal);	/* IMP: R-31275-44060 */
 	} else {
-		sqlVdbeMemStringify(pVal);
+		mem_convert_to_string(pVal);
 		assert(0 == (1 & SQL_PTR_TO_INT(pVal->z)));
 	}
 	return pVal->z;
@@ -1978,74 +2103,6 @@ sqlVdbeMemCast(Mem * pMem, enum field_type type)
 	}
 }
 
-/*
- * Add MEM_Str to the set of representations for the given Mem.  Numbers
- * are converted using sql_snprintf().  Converting a BLOB to a string
- * is a no-op.
- *
- * Existing representations MEM_Int and MEM_Real are invalidated if
- * bForce is true but are retained if bForce is false.
- *
- * A MEM_Null value will never be passed to this function. This function is
- * used for converting values to text for returning to the user (i.e. via
- * sql_value_text()), or for ensuring that values to be used as btree
- * keys are strings. In the former case a NULL pointer is returned the
- * user and the latter is an internal programming error.
- */
-int
-sqlVdbeMemStringify(Mem * pMem)
-{
-	int fg = pMem->flags;
-	int nByte = 32;
-
-	if ((fg & (MEM_Null | MEM_Str | MEM_Blob)) != 0 &&
-	    !mem_has_msgpack_subtype(pMem))
-		return 0;
-
-	assert(!(fg & MEM_Zero));
-	assert((fg & (MEM_Int | MEM_UInt | MEM_Real | MEM_Bool |
-		      MEM_Blob)) != 0);
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-
-	/*
-	 * In case we have ARRAY/MAP we should save decoded value
-	 * before clearing pMem->z.
-	 */
-	char *value = NULL;
-	if (mem_has_msgpack_subtype(pMem)) {
-		const char *value_str = mp_str(pMem->z);
-		nByte = strlen(value_str) + 1;
-		value = region_alloc(&fiber()->gc, nByte);
-		memcpy(value, value_str, nByte);
-	}
-
-	if (sqlVdbeMemClearAndResize(pMem, nByte)) {
-		return -1;
-	}
-	if (fg & MEM_Int) {
-		sql_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
-		pMem->flags &= ~MEM_Int;
-	} else if ((fg & MEM_UInt) != 0) {
-		sql_snprintf(nByte, pMem->z, "%llu", pMem->u.u);
-		pMem->flags &= ~MEM_UInt;
-	} else if ((fg & MEM_Bool) != 0) {
-		sql_snprintf(nByte, pMem->z, "%s",
-			     SQL_TOKEN_BOOLEAN(pMem->u.b));
-		pMem->flags &= ~MEM_Bool;
-	} else if (mem_has_msgpack_subtype(pMem)) {
-		sql_snprintf(nByte, pMem->z, "%s", value);
-		pMem->flags &= ~MEM_Subtype;
-		pMem->subtype = SQL_SUBTYPE_NO;
-	} else {
-		assert(fg & MEM_Real);
-		sql_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
-		pMem->flags &= ~MEM_Real;
-	}
-	pMem->n = sqlStrlen30(pMem->z);
-	pMem->flags |= MEM_Str | MEM_Term;
-	return 0;
-}
-
 /*
  * Make sure the given Mem is \u0000 terminated.
  */
@@ -2163,7 +2220,7 @@ mem_apply_type(struct Mem *record, enum field_type type)
 		 */
 		if ((record->flags & MEM_Str) == 0 &&
 		    (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
-			sqlVdbeMemStringify(record);
+			mem_convert_to_string(record);
 		record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
 		return 0;
 	case FIELD_TYPE_VARBINARY:
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d5f0d7891..d370b448b 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -330,6 +330,12 @@ mem_convert_to_double(struct Mem *mem);
 int
 mem_convert_to_number(struct Mem *mem);
 
+int
+mem_convert_to_string(struct Mem *mem);
+
+int
+mem_convert_to_string0(struct Mem *mem);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -381,7 +387,6 @@ registerTrace(int iReg, Mem *p);
 #endif
 
 int sqlVdbeMemCast(struct Mem *, enum field_type type);
-int sqlVdbeMemStringify(struct Mem *);
 int sqlVdbeMemNulTerminate(struct Mem *);
 int sqlVdbeMemExpandBlob(struct Mem *);
 #define ExpandBlob(P) (mem_is_zeroblob(P)? sqlVdbeMemExpandBlob(P) : 0)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 45/53] sql: introduce mem_explicit_cast()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (43 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 44/53] sql: introduce mem_convert_to_string() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 46/53] sql: introduce mem_implicit_cast() Mergen Imeev via Tarantool-patches
                   ` (7 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_explicit_cast(). This function is used to
convert a MEM to a given field type according to explicit cast rules.

Part of #5818
---
 src/box/sql/mem.c  | 302 +++++++++++++++++++--------------------------
 src/box/sql/mem.h  |  15 +--
 src/box/sql/vdbe.c |   4 +-
 3 files changed, 133 insertions(+), 188 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 262f48aca..559bf6121 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -943,6 +943,134 @@ mem_convert_to_string(struct Mem *mem)
 	return -1;
 }
 
+static inline int
+mem_convert_varstring_to_unsigned(struct Mem *mem)
+{
+	bool is_neg;
+	int64_t i;
+	if (sql_atoi64(mem->z, &i, &is_neg, mem->n) != 0)
+		return -1;
+	if (is_neg)
+		return -1;
+	mem_set_unsigned(mem, (uint64_t)i);
+	return 0;
+}
+
+static inline int
+mem_convert_boolean_to_unsigned(struct Mem *mem)
+{
+	mem_set_unsigned(mem, (uint64_t)mem->u.b);
+	return 0;
+}
+
+static inline int
+mem_convert_string_to_boolean(struct Mem *mem)
+{
+	char *str = mem->z;
+	bool b;
+	const char *str_true = "TRUE";
+	const char *str_false = "FALSE";
+	uint32_t len_true = strlen(str_true);
+	uint32_t len_false = strlen(str_false);
+
+	for (; str[0] == ' '; str++);
+	if (strncasecmp(str, str_true, len_true) == 0) {
+		b = true;
+		str += len_true;
+	} else if (strncasecmp(str, str_false, len_false) == 0) {
+		b = false;
+		str += len_false;
+	} else {
+		return -1;
+	}
+	for (; str[0] == ' '; str++);
+	if (str[0] != '\0')
+		return -1;
+	mem_set_boolean(mem, b);
+	return 0;
+}
+
+static inline int
+mem_convert_integer_to_boolean(struct Mem *mem)
+{
+	mem_set_boolean(mem, mem->u.u != 0);
+	return 0;
+}
+
+static inline int
+mem_convert_double_to_boolean(struct Mem *mem)
+{
+	mem_set_boolean(mem, mem->u.r != 0);
+	return 0;
+}
+
+static inline int
+mem_convert_string_to_varbinary(struct Mem *mem)
+{
+	mem->flags = (mem->flags & (MEM_Dyn | MEM_Static | MEM_Ephem)) |
+		     MEM_Blob;
+	mem->field_type = FIELD_TYPE_VARBINARY;
+	return 0;
+}
+
+int
+mem_explicit_cast(struct Mem *mem, enum field_type type)
+{
+	if ((mem->flags & MEM_Null) != 0) {
+		mem->field_type = type;
+		return 0;
+	}
+	switch (type) {
+	case FIELD_TYPE_UNSIGNED:
+		if ((mem->flags & MEM_UInt) != 0)
+			return 0;
+		if ((mem->flags & MEM_Int) != 0)
+			return -1;
+		if ((mem->flags & MEM_Blob) != 0 &&
+		    (mem->flags & MEM_Subtype) != 0)
+			return -1;
+		if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+			return mem_convert_varstring_to_unsigned(mem);
+		if ((mem->flags & MEM_Real) != 0)
+			return mem_convert_double_to_integer(mem);
+		if ((mem->flags & MEM_Bool) != 0)
+			return mem_convert_boolean_to_unsigned(mem);
+		return -1;
+	case FIELD_TYPE_STRING:
+		return mem_convert_to_string(mem);
+	case FIELD_TYPE_DOUBLE:
+		return mem_convert_to_double(mem);
+	case FIELD_TYPE_INTEGER:
+		return mem_convert_to_integer(mem);
+	case FIELD_TYPE_BOOLEAN:
+		if ((mem->flags & MEM_Bool) != 0)
+			return 0;
+		if ((mem->flags & (MEM_UInt | MEM_Int)) != 0)
+			return mem_convert_integer_to_boolean(mem);
+		if ((mem->flags & MEM_Str) != 0)
+			return mem_convert_string_to_boolean(mem);
+		if ((mem->flags & MEM_Real) != 0)
+			return mem_convert_double_to_boolean(mem);
+		return -1;
+	case FIELD_TYPE_VARBINARY:
+		if (mem_is_binary(mem))
+			return 0;
+		if (mem_is_string(mem))
+			return mem_convert_string_to_varbinary(mem);
+		return -1;
+	case FIELD_TYPE_NUMBER:
+		return mem_convert_to_number(mem);
+	case FIELD_TYPE_SCALAR:
+		if ((mem->flags & MEM_Blob) != 0 &&
+		    (mem->flags & MEM_Subtype) != 0)
+			return -1;
+		return 0;
+	default:
+		break;
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1583,42 +1711,6 @@ valueToText(sql_value * pVal)
 	return pVal->z;
 }
 
-/**
- * According to ANSI SQL string value can be converted to boolean
- * type if string consists of literal "true" or "false" and
- * number of leading and trailing spaces.
- *
- * For instance, "   tRuE  " can be successfully converted to
- * boolean value true.
- *
- * @param str String to be converted to boolean. Assumed to be
- *        null terminated.
- * @param[out] result Resulting value of cast.
- * @retval 0 If string satisfies conditions above.
- * @retval -1 Otherwise.
- */
-static int
-str_cast_to_boolean(const char *str, bool *result)
-{
-	assert(str != NULL);
-	for (; *str == ' '; str++);
-	if (strncasecmp(str, SQL_TOKEN_TRUE, strlen(SQL_TOKEN_TRUE)) == 0) {
-		*result = true;
-		str += 4;
-	} else if (strncasecmp(str, SQL_TOKEN_FALSE,
-			       strlen(SQL_TOKEN_FALSE)) == 0) {
-		*result = false;
-		str += 5;
-	} else {
-		return -1;
-	}
-	for (; *str != '\0'; ++str) {
-		if (*str != ' ')
-			return -1;
-	}
-	return 0;
-}
-
 /*
  * Convert a 64-bit IEEE double into a 64-bit signed integer.
  * If the double is out of range of a 64-bit signed integer then
@@ -2000,109 +2092,6 @@ registerTrace(int iReg, Mem *p) {
 }
 #endif
 
-/*
- * Cast the datatype of the value in pMem according to the type
- * @type.  Casting is different from applying type in that a cast
- * is forced.  In other words, the value is converted into the desired
- * type even if that results in loss of data.  This routine is
- * used (for example) to implement the SQL "cast()" operator.
- */
-int
-sqlVdbeMemCast(Mem * pMem, enum field_type type)
-{
-	assert(type < field_type_MAX);
-	if (pMem->flags & MEM_Null)
-		return 0;
-	switch (type) {
-	case FIELD_TYPE_SCALAR:
-		return 0;
-	case FIELD_TYPE_BOOLEAN:
-		if ((pMem->flags & MEM_Int) != 0) {
-			mem_set_boolean(pMem, pMem->u.i);
-			return 0;
-		}
-		if ((pMem->flags & MEM_UInt) != 0) {
-			mem_set_boolean(pMem, pMem->u.u);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Real) != 0) {
-			mem_set_boolean(pMem, pMem->u.r);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Str) != 0) {
-			bool value;
-			if (str_cast_to_boolean(pMem->z, &value) != 0)
-				return -1;
-			mem_set_boolean(pMem, value);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Bool) != 0)
-			return 0;
-		return -1;
-	case FIELD_TYPE_INTEGER:
-	case FIELD_TYPE_UNSIGNED:
-		if ((pMem->flags & (MEM_Blob | MEM_Str)) != 0) {
-			bool is_neg;
-			int64_t val;
-			if (sql_atoi64(pMem->z, &val, &is_neg, pMem->n) != 0)
-				return -1;
-			if (type == FIELD_TYPE_UNSIGNED && is_neg)
-				return -1;
-			mem_set_integer(pMem, val, is_neg);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Bool) != 0) {
-			mem_set_integer(pMem, (int64_t)pMem->u.b, false);
-			return 0;
-		}
-		if ((pMem->flags & MEM_Real) != 0) {
-			double d;
-			if (sqlVdbeRealValue(pMem, &d) != 0)
-				return -1;
-			if (d < (double)INT64_MAX && d >= (double)INT64_MIN) {
-				mem_set_integer(pMem, d, d <= -1);
-				return 0;
-			}
-			if (d >= (double)INT64_MAX && d < (double)UINT64_MAX) {
-				mem_set_unsigned(pMem, d);
-				return 0;
-			}
-			return -1;
-		}
-		if (type == FIELD_TYPE_UNSIGNED &&
-		    (pMem->flags & MEM_UInt) == 0)
-			return -1;
-		return 0;
-	case FIELD_TYPE_DOUBLE:
-		return mem_convert_to_double(pMem);
-	case FIELD_TYPE_NUMBER:
-		return mem_convert_to_number(pMem);
-	case FIELD_TYPE_VARBINARY:
-		if ((pMem->flags & MEM_Blob) != 0)
-			return 0;
-		if ((pMem->flags & MEM_Str) != 0) {
-			MemSetTypeFlag(pMem, MEM_Str);
-			return 0;
-		}
-		return -1;
-	default:
-		assert(type == FIELD_TYPE_STRING);
-		assert(MEM_Str == (MEM_Blob >> 3));
-		if ((pMem->flags & MEM_Bool) != 0) {
-			const char *str_bool = SQL_TOKEN_BOOLEAN(pMem->u.b);
-			if (mem_copy_string0(pMem, str_bool) != 0)
-				return -1;
-			return 0;
-		}
-		pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
-			sql_value_apply_type(pMem, FIELD_TYPE_STRING);
-		assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
-		pMem->flags &=
-			~(MEM_Int | MEM_UInt | MEM_Real | MEM_Blob | MEM_Zero);
-		return 0;
-	}
-}
-
 /*
  * Make sure the given Mem is \u0000 terminated.
  */
@@ -2255,41 +2244,6 @@ mem_apply_type(struct Mem *record, enum field_type type)
 	}
 }
 
-/**
- * Convert the numeric value contained in MEM to unsigned.
- *
- * @param mem The MEM that contains the numeric value.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-static int
-mem_convert_to_unsigned(struct Mem *mem)
-{
-	if ((mem->flags & MEM_UInt) != 0)
-		return 0;
-	if ((mem->flags & MEM_Int) != 0)
-		return -1;
-	if ((mem->flags & MEM_Real) == 0)
-		return -1;
-	double d = mem->u.r;
-	if (d < 0.0 || d >= (double)UINT64_MAX)
-		return -1;
-	mem_set_unsigned(mem, (uint64_t) d);
-	return 0;
-}
-
-int
-mem_convert_to_numeric(struct Mem *mem, enum field_type type)
-{
-	assert(mem_is_number(mem) && sql_type_is_numeric(type));
-	assert(type != FIELD_TYPE_NUMBER);
-	if (type == FIELD_TYPE_DOUBLE)
-		return mem_convert_to_double(mem);
-	if (type == FIELD_TYPE_UNSIGNED)
-		return mem_convert_to_unsigned(mem);
-	assert(type == FIELD_TYPE_INTEGER);
-	return mem_convert_to_integer(mem);
-}
-
 /*
  * Make sure pMem->z points to a writable allocation of at least
  * min(n,32) bytes.
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index d370b448b..922dad272 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -336,6 +336,9 @@ mem_convert_to_string(struct Mem *mem);
 int
 mem_convert_to_string0(struct Mem *mem);
 
+int
+mem_explicit_cast(struct Mem *mem, enum field_type type);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -386,7 +389,6 @@ registerTrace(int iReg, Mem *p);
 # define memAboutToChange(P,M)
 #endif
 
-int sqlVdbeMemCast(struct Mem *, enum field_type type);
 int sqlVdbeMemNulTerminate(struct Mem *);
 int sqlVdbeMemExpandBlob(struct Mem *);
 #define ExpandBlob(P) (mem_is_zeroblob(P)? sqlVdbeMemExpandBlob(P) : 0)
@@ -429,17 +431,6 @@ void sql_value_apply_type(struct Mem *val, enum field_type type);
 int
 mem_apply_type(struct Mem *record, enum field_type type);
 
-/**
- * Convert the numeric value contained in MEM to another numeric
- * type.
- *
- * @param mem The MEM that contains the numeric value.
- * @param type The type to convert to.
- * @retval 0 if the conversion was successful, -1 otherwise.
- */
-int
-mem_convert_to_numeric(struct Mem *mem, enum field_type type);
-
 /** Setters = Change MEM value. */
 
 int sqlVdbeMemGrow(struct Mem * pMem, int n, int preserve);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 6799cc9aa..a567f69bd 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -1389,7 +1389,7 @@ case OP_Cast: {                  /* in1 */
 	pIn1 = &aMem[pOp->p1];
 	if (ExpandBlob(pIn1) != 0)
 		goto abort_due_to_error;
-	rc = sqlVdbeMemCast(pIn1, pOp->p2);
+	rc = mem_explicit_cast(pIn1, pOp->p2);
 	/*
 	 * SCALAR is not type itself, but rather an aggregation
 	 * of types. Hence, cast to this type shouldn't change
@@ -2009,7 +2009,7 @@ case OP_ApplyType: {
 			if (!mem_is_number(pIn1))
 				goto type_mismatch;
 			/* Try to convert numeric-to-numeric. */
-			if (mem_convert_to_numeric(pIn1, type) != 0)
+			if (mem_explicit_cast(pIn1, type) != 0)
 				goto type_mismatch;
 		}
 		pIn1++;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 46/53] sql: introduce mem_implicit_cast()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (44 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 45/53] sql: introduce mem_explicit_cast() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 47/53] sql: introduce mem_get_integer() Mergen Imeev via Tarantool-patches
                   ` (6 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_implicit_cast(). This function is used to
convert a MEM to given type according to implicit cast rules.

Part of #5818
---
 src/box/sql/mem.c     | 275 ++++++++++++++++++++++++------------------
 src/box/sql/mem.h     |  55 +--------
 src/box/sql/vdbe.c    |  21 +---
 src/box/sql/vdbeaux.c |   2 +-
 4 files changed, 168 insertions(+), 185 deletions(-)

diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 559bf6121..1baf4c482 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -943,6 +943,28 @@ mem_convert_to_string(struct Mem *mem)
 	return -1;
 }
 
+static inline int
+mem_convert_double_to_unsigned(struct Mem *mem)
+{
+	double d = mem->u.r;
+	if (d >= 0 && d < (double)UINT64_MAX) {
+		mem_set_unsigned(mem, (uint64_t)d);
+		return 0;
+	}
+	return -1;
+}
+
+static inline int
+mem_convert_double_to_unsigned_lossless(struct Mem *mem)
+{
+	double d = mem->u.r;
+	if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
+		mem_set_unsigned(mem, (uint64_t)d);
+		return 0;
+	}
+	return -1;
+}
+
 static inline int
 mem_convert_varstring_to_unsigned(struct Mem *mem)
 {
@@ -1071,6 +1093,140 @@ mem_explicit_cast(struct Mem *mem, enum field_type type)
 	return -1;
 }
 
+int
+mem_implicit_cast(struct Mem *mem, enum field_type type)
+{
+	if ((mem->flags & MEM_Null) != 0) {
+		mem->field_type = type;
+		return 0;
+	}
+	switch (type) {
+	case FIELD_TYPE_UNSIGNED:
+		if ((mem->flags & MEM_UInt) != 0)
+			return 0;
+		if ((mem->flags & MEM_Real) != 0)
+			return mem_convert_double_to_unsigned(mem);
+		return -1;
+	case FIELD_TYPE_STRING:
+		if ((mem->flags & MEM_Str) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_DOUBLE:
+		if ((mem->flags & MEM_Real) != 0)
+			return 0;
+		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+			return mem_convert_integer_to_double(mem);
+		return -1;
+	case FIELD_TYPE_INTEGER:
+		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+			return 0;
+		if ((mem->flags & MEM_Real) != 0)
+			return mem_convert_double_to_integer(mem);
+		return -1;
+	case FIELD_TYPE_BOOLEAN:
+		if ((mem->flags & MEM_Bool) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_VARBINARY:
+		if ((mem->flags & MEM_Blob) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_NUMBER:
+		if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_MAP:
+		if (mem_is_map(mem))
+			return 0;
+		return -1;
+	case FIELD_TYPE_ARRAY:
+		if (mem_is_array(mem))
+			return 0;
+		return -1;
+	case FIELD_TYPE_SCALAR:
+		if ((mem->flags & MEM_Blob) != 0 &&
+		    (mem->flags & MEM_Subtype) != 0)
+			return -1;
+		return 0;
+	case FIELD_TYPE_ANY:
+		return 0;
+	default:
+		break;
+	}
+	return -1;
+}
+
+int
+mem_implicit_cast_old(struct Mem *mem, enum field_type type)
+{
+	if (mem_is_null(mem))
+		return 0;
+	switch (type) {
+	case FIELD_TYPE_UNSIGNED:
+		if ((mem->flags & MEM_UInt) != 0)
+			return 0;
+		if ((mem->flags & MEM_Real) != 0)
+			return mem_convert_double_to_unsigned_lossless(mem);
+		if ((mem->flags & MEM_Str) != 0)
+			return mem_convert_varstring_to_unsigned(mem);
+		return -1;
+	case FIELD_TYPE_STRING:
+		if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+			return 0;
+		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+			return mem_convert_integer_to_string(mem);
+		if ((mem->flags & MEM_Real) != 0)
+			return mem_convert_double_to_string(mem);
+		return -1;
+	case FIELD_TYPE_DOUBLE:
+		if ((mem->flags & MEM_Real) != 0)
+			return 0;
+		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+			return mem_convert_integer_to_double(mem);
+		if ((mem->flags & MEM_Str) != 0)
+			return mem_convert_varstring_to_double(mem);
+		return -1;
+	case FIELD_TYPE_INTEGER:
+		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
+			return 0;
+		if ((mem->flags & MEM_Str) != 0)
+			return mem_convert_varstring_to_integer(mem);
+		if (mem_is_double(mem))
+			return mem_convert_double_to_integer_lossless(mem);
+		return -1;
+	case FIELD_TYPE_BOOLEAN:
+		if ((mem->flags & MEM_Bool) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_VARBINARY:
+		if ((mem->flags & MEM_Blob) != 0)
+			return 0;
+		return -1;
+	case FIELD_TYPE_NUMBER:
+		if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
+			return 0;
+		if ((mem->flags & MEM_Str) != 0)
+			return mem_convert_to_number(mem);
+		return -1;
+	case FIELD_TYPE_MAP:
+		if (mem_is_map(mem))
+			return 0;
+		return -1;
+	case FIELD_TYPE_ARRAY:
+		if (mem_is_array(mem))
+			return 0;
+		return -1;
+	case FIELD_TYPE_SCALAR:
+		if ((mem->flags & MEM_Blob) != 0 &&
+		    (mem->flags & MEM_Subtype) != 0)
+			return -1;
+		return 0;
+	default:
+		break;
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2133,117 +2289,6 @@ sqlVdbeMemExpandBlob(Mem * pMem)
 	return 0;
 }
 
-/*
- * Exported version of mem_apply_type(). This one works on sql_value*,
- * not the internal Mem* type.
- */
-void
-sql_value_apply_type(
-	sql_value *pVal,
-	enum field_type type)
-{
-	mem_apply_type((Mem *) pVal, type);
-}
-
-int
-mem_apply_type(struct Mem *record, enum field_type type)
-{
-	if ((record->flags & MEM_Null) != 0)
-		return 0;
-	assert(type < field_type_MAX);
-	switch (type) {
-	case FIELD_TYPE_INTEGER:
-	case FIELD_TYPE_UNSIGNED:
-		if ((record->flags & (MEM_Bool | MEM_Blob)) != 0)
-			return -1;
-		if ((record->flags & MEM_UInt) == MEM_UInt)
-			return 0;
-		if ((record->flags & MEM_Real) == MEM_Real) {
-			double d = record->u.r;
-			if (d >= 0) {
-				if (double_compare_uint64(d, UINT64_MAX,
-							  1) > 0)
-					return 0;
-				if ((double)(uint64_t)d == d)
-					mem_set_unsigned(record, (uint64_t)d);
-			} else {
-				if (double_compare_nint64(d, INT64_MIN, 1) < 0)
-					return 0;
-				if ((double)(int64_t)d == d) {
-					mem_set_integer(record, (int64_t)d,
-							true);
-				}
-			}
-			return 0;
-		}
-		if ((record->flags & MEM_Str) != 0) {
-			bool is_neg;
-			int64_t i;
-			if (sql_atoi64(record->z, &i, &is_neg, record->n) != 0)
-				return -1;
-			mem_set_integer(record, i, is_neg);
-		}
-		if ((record->flags & MEM_Int) == MEM_Int) {
-			if (type == FIELD_TYPE_UNSIGNED)
-				return -1;
-			return 0;
-		}
-		return 0;
-	case FIELD_TYPE_BOOLEAN:
-		if ((record->flags & MEM_Bool) == MEM_Bool)
-			return 0;
-		return -1;
-	case FIELD_TYPE_NUMBER:
-		if ((record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
-			return 0;
-		return mem_convert_to_double(record);
-	case FIELD_TYPE_DOUBLE:
-		if ((record->flags & MEM_Real) != 0)
-			return 0;
-		return mem_convert_to_double(record);
-	case FIELD_TYPE_STRING:
-		/*
-		 * Only attempt the conversion to TEXT if there is
-		 * an integer or real representation (BLOB and
-		 * NULL do not get converted).
-		 */
-		if ((record->flags & MEM_Str) == 0 &&
-		    (record->flags & (MEM_Real | MEM_Int | MEM_UInt)) != 0)
-			mem_convert_to_string(record);
-		record->flags &= ~(MEM_Real | MEM_Int | MEM_UInt);
-		return 0;
-	case FIELD_TYPE_VARBINARY:
-		if ((record->flags & MEM_Blob) == 0)
-			return -1;
-		return 0;
-	case FIELD_TYPE_SCALAR:
-		/* Can't cast MAP and ARRAY to scalar types. */
-		if ((record->flags & MEM_Subtype) != 0 &&
-		    record->subtype == SQL_SUBTYPE_MSGPACK) {
-			assert(mp_typeof(*record->z) == MP_MAP ||
-			       mp_typeof(*record->z) == MP_ARRAY);
-			return -1;
-		}
-		return 0;
-	case FIELD_TYPE_MAP:
-		if ((record->flags & MEM_Subtype) != 0 &&
-		    record->subtype == SQL_SUBTYPE_MSGPACK &&
-		    mp_typeof(*record->z) == MP_MAP)
-			return 0;
-		return -1;
-	case FIELD_TYPE_ARRAY:
-		if ((record->flags & MEM_Subtype) != 0 &&
-		    record->subtype == SQL_SUBTYPE_MSGPACK &&
-		    mp_typeof(*record->z) == MP_ARRAY)
-			return 0;
-		return -1;
-	case FIELD_TYPE_ANY:
-		return 0;
-	default:
-		return -1;
-	}
-}
-
 /*
  * Make sure pMem->z points to a writable allocation of at least
  * min(n,32) bytes.
@@ -2769,14 +2814,6 @@ sqlMemCompare(const Mem * pMem1, const Mem * pMem2, const struct coll * pColl)
 	return sqlBlobCompare(pMem1, pMem2);
 }
 
-bool
-mem_is_type_compatible(struct Mem *mem, enum field_type type)
-{
-	enum mp_type mp_type = mem_mp_type(mem);
-	assert(mp_type < MP_EXT);
-	return field_mp_plain_type_is_compatible(type, mp_type, true);
-}
-
 int
 sql_vdbemem_finalize(struct Mem *mem, struct func *func)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 922dad272..c0b9f4f99 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -339,6 +339,12 @@ mem_convert_to_string0(struct Mem *mem);
 int
 mem_explicit_cast(struct Mem *mem, enum field_type type);
 
+int
+mem_implicit_cast(struct Mem *mem, enum field_type type);
+
+int
+mem_implicit_cast_old(struct Mem *mem, enum field_type type);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -392,44 +398,6 @@ registerTrace(int iReg, Mem *p);
 int sqlVdbeMemNulTerminate(struct Mem *);
 int sqlVdbeMemExpandBlob(struct Mem *);
 #define ExpandBlob(P) (mem_is_zeroblob(P)? sqlVdbeMemExpandBlob(P) : 0)
-void sql_value_apply_type(struct Mem *val, enum field_type type);
-
-
-/**
- * Processing is determined by the field type parameter:
- *
- * INTEGER:
- *    If memory holds floating point value and it can be
- *    converted without loss (2.0 - > 2), it's type is
- *    changed to INT. Otherwise, simply return success status.
- *
- * NUMBER:
- *    If memory holds INT or floating point value,
- *    no actions take place.
- *
- * STRING:
- *    Convert mem to a string representation.
- *
- * SCALAR:
- *    Mem is unchanged, but flag is set to BLOB in case of
- *    scalar-like type. Otherwise, (MAP, ARRAY) conversion
- *    is impossible.
- *
- * BOOLEAN:
- *    If memory holds BOOLEAN no actions take place.
- *
- * ANY:
- *    Mem is unchanged, no actions take place.
- *
- * MAP/ARRAY:
- *    These types can't be casted to scalar ones, or to each
- *    other. So the only valid conversion is to type itself.
- *
- * @param record The value to apply type to.
- * @param type The type to be applied.
- */
-int
-mem_apply_type(struct Mem *record, enum field_type type);
 
 /** Setters = Change MEM value. */
 
@@ -494,17 +462,6 @@ int sqlVdbeMemTooBig(Mem *);
 
 int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
 
-/**
- * Check that MEM_type of the mem is compatible with given type.
- *
- * @param mem The MEM that contains the value to check.
- * @param type The type to check.
- * @retval TRUE if the MEM_type of the value and the given type
- *         are compatible, FALSE otherwise.
- */
-bool
-mem_is_type_compatible(struct Mem *mem, enum field_type type);
-
 /** MEM manipulate functions. */
 
 /**
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index a567f69bd..389472941 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2001,23 +2001,12 @@ case OP_ApplyType: {
 	while((type = *(types++)) != field_type_MAX) {
 		assert(pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)]);
 		assert(memIsValid(pIn1));
-		if (!mem_is_type_compatible(pIn1, type)) {
-			/* Implicit cast is allowed only to numeric type. */
-			if (!sql_type_is_numeric(type))
-				goto type_mismatch;
-			/* Implicit cast is allowed only from numeric type. */
-			if (!mem_is_number(pIn1))
-				goto type_mismatch;
-			/* Try to convert numeric-to-numeric. */
-			if (mem_explicit_cast(pIn1, type) != 0)
-				goto type_mismatch;
+		if (mem_implicit_cast(pIn1, type) != 0) {
+			diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
+				 mem_str(pIn1), field_type_strs[type]);
+			goto abort_due_to_error;
 		}
 		pIn1++;
-		continue;
-type_mismatch:
-		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
-			 mem_str(pIn1), field_type_strs[type]);
-		goto abort_due_to_error;
 	}
 	break;
 }
@@ -2076,7 +2065,7 @@ case OP_MakeRecord: {
 	if (types != NULL) {
 		pRec = pData0;
 		do {
-			mem_apply_type(pRec++, *(types++));
+			mem_implicit_cast_old(pRec++, *(types++));
 		} while(types[0] != field_type_MAX);
 	}
 
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index ccabf6035..59fc313d4 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -2310,7 +2310,7 @@ sqlVdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
 			sql_value *pRet = sqlValueNew(v->db);
 			if (pRet) {
 				mem_copy(pRet, pMem);
-				sql_value_apply_type(pRet, aff);
+				mem_implicit_cast_old(pRet, aff);
 			}
 			return pRet;
 		}
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 47/53] sql: introduce mem_get_integer()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (45 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 46/53] sql: introduce mem_implicit_cast() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 48/53] sql: introduce mem_get_unsigned() Mergen Imeev via Tarantool-patches
                   ` (5 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_integer() which is used to receive integer
value from MEM.

Part of #5818
---
 src/box/sql/func.c    |  55 ++++++++++++-------
 src/box/sql/mem.c     | 123 +++++++++++++-----------------------------
 src/box/sql/mem.h     |  10 ++--
 src/box/sql/printf.c  |   5 +-
 src/box/sql/sqlInt.h  |   6 ---
 src/box/sql/vdbeapi.c |  12 -----
 6 files changed, 80 insertions(+), 131 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index b644c39d8..0fa0f6ac7 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -111,9 +111,13 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 		sql_value *param =
 			(sql_value *)((struct Mem *)port->mem + i);
 		switch (sql_value_type(param)) {
-		case MP_INT:
-			luaL_pushint64(L, sql_value_int64(param));
+		case MP_INT: {
+			bool unused;
+			int64_t n;
+			mem_get_integer(param, &n, &unused);
+			luaL_pushint64(L, n);
 			break;
+		}
 		case MP_UINT:
 			luaL_pushuint64(L, sql_value_uint64(param));
 			break;
@@ -157,7 +161,9 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 			(sql_value *)((struct Mem *)port->mem + i);
 		switch (sql_value_type(param)) {
 		case MP_INT: {
-			sql_int64 val = sql_value_int64(param);
+			bool unused;
+			int64_t val;
+			mem_get_integer(param, &val, &unused);
 			if (val < 0) {
 				mpstream_encode_int(&stream, val);
 				break;
@@ -509,7 +515,9 @@ absFunc(sql_context * context, int argc, sql_value ** argv)
 		break;
 	}
 	case MP_INT: {
-		int64_t value = sql_value_int64(argv[0]);
+		bool unused;
+		int64_t value;
+		mem_get_integer(argv[0], &value, &unused);
 		assert(value < 0);
 		sql_result_uint(context, -value);
 		break;
@@ -725,7 +733,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 	const unsigned char *z2;
 	int len;
 	int p0type;
-	i64 p1, p2;
+	int64_t p1, p2;
 	int negP2 = 0;
 
 	if (argc != 2 && argc != 3) {
@@ -740,7 +748,8 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 		return;
 	}
 	p0type = sql_value_type(argv[0]);
-	p1 = sql_value_int(argv[1]);
+	bool unused;
+	mem_get_integer(argv[1], &p1, &unused);
 	if (p0type == MP_BIN) {
 		len = sql_value_bytes(argv[0]);
 		z = sql_value_blob(argv[0]);
@@ -756,7 +765,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 			len = sql_utf8_char_count(z, sql_value_bytes(argv[0]));
 	}
 	if (argc == 3) {
-		p2 = sql_value_int(argv[2]);
+		mem_get_integer(argv[2], &p2, &unused);
 		if (p2 < 0) {
 			p2 = -p2;
 			negP2 = 1;
@@ -827,7 +836,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 static void
 roundFunc(sql_context * context, int argc, sql_value ** argv)
 {
-	int n = 0;
+	int64_t n = 0;
 	double r;
 	if (argc != 1 && argc != 2) {
 		diag_set(ClientError, ER_FUNC_WRONG_ARG_COUNT, "ROUND",
@@ -838,7 +847,8 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
 	if (argc == 2) {
 		if (mem_is_null(argv[1]))
 			return;
-		n = sql_value_int(argv[1]);
+		bool unused;
+		mem_get_integer(argv[1], &n, &unused);
 		if (n < 0)
 			n = 0;
 	}
@@ -981,7 +991,7 @@ randomFunc(sql_context * context, int NotUsed, sql_value ** NotUsed2)
 static void
 randomBlob(sql_context * context, int argc, sql_value ** argv)
 {
-	int n;
+	int64_t n;
 	unsigned char *p;
 	assert(argc == 1);
 	UNUSED_PARAMETER(argc);
@@ -991,7 +1001,8 @@ randomBlob(sql_context * context, int argc, sql_value ** argv)
 		context->is_aborted = true;
 		return;
 	}
-	n = sql_value_int(argv[0]);
+	bool unused;
+	mem_get_integer(argv[0], &n, &unused);
 	if (n < 1)
 		return;
 	p = contextMalloc(context, n);
@@ -1532,10 +1543,11 @@ hexFunc(sql_context * context, int argc, sql_value ** argv)
 static void
 zeroblobFunc(sql_context * context, int argc, sql_value ** argv)
 {
-	i64 n;
+	int64_t n;
 	assert(argc == 1);
 	UNUSED_PARAMETER(argc);
-	n = sql_value_int64(argv[0]);
+	bool unused;
+	mem_get_integer(argv[0], &n, &unused);
 	if (n < 0)
 		n = 0;
 	if (sql_result_zeroblob64(context, n) != 0) {
@@ -1779,9 +1791,11 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1,
 	int input_str_sz = sql_value_bytes(arg2);
 	if (sql_value_type(arg1) == MP_INT || sql_value_type(arg1) == MP_UINT) {
 		uint8_t len_one = 1;
-		trim_procedure(context, sql_value_int(arg1),
-			       (const unsigned char *) " ", &len_one, 1,
-			       input_str, input_str_sz);
+		bool unused;
+		int64_t n;
+		mem_get_integer(arg1, &n, &unused);
+		trim_procedure(context, n, (const unsigned char *) " ",
+			       &len_one, 1, input_str, input_str_sz);
 	} else if ((trim_set = sql_value_text(arg1)) != NULL) {
 		int trim_set_sz = sql_value_bytes(arg1);
 		uint8_t *char_len;
@@ -1819,7 +1833,10 @@ trim_func_three_args(struct sql_context *context, sql_value *arg1,
 					     &char_len);
 	if (char_cnt == -1)
 		return;
-	trim_procedure(context, sql_value_int(arg1), trim_set, char_len,
+	bool unused;
+	int64_t n;
+	mem_get_integer(arg1, &n, &unused);
+	trim_procedure(context, n, trim_set, char_len,
 		       char_cnt, input_str, input_str_sz);
 	sql_free(char_len);
 }
@@ -1958,7 +1975,9 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
 	}
 	p->cnt++;
 	if (type == MP_INT || type == MP_UINT) {
-		int64_t v = sql_value_int64(argv[0]);
+		bool unused;
+		int64_t v;
+		mem_get_integer(argv[0], &v, &unused);
 		if (type == MP_INT)
 			p->rSum += v;
 		else
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 1baf4c482..540931723 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1227,6 +1227,38 @@ mem_implicit_cast_old(struct Mem *mem, enum field_type type)
 	return -1;
 }
 
+int
+mem_get_integer(const struct Mem *mem, int64_t *i, bool *is_neg)
+{
+	if ((mem->flags & MEM_Int) != 0) {
+		*i = mem->u.i;
+		*is_neg = true;
+		return 0;
+	}
+	if ((mem->flags & MEM_UInt) != 0) {
+		*i = mem->u.i;
+		*is_neg = false;
+		return 0;
+	}
+	if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
+		return sql_atoi64(mem->z, i, is_neg, mem->n);
+	if ((mem->flags & MEM_Real) != 0) {
+		double d = mem->u.r;
+		if (d < 0 && d >= (double)INT64_MIN) {
+			*i = (int64_t)d;
+			*is_neg = true;
+			return 0;
+		}
+		if (d >= 0 && d < (double)UINT64_MAX) {
+			*i = (int64_t)(uint64_t)d;
+			*is_neg = false;
+			return 0;
+		}
+		return -1;
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1536,12 +1568,12 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
 	int64_t r;
 	int64_t res;
 	bool unused;
-	if (sqlVdbeIntValue(left, &l, &unused) != 0) {
+	if (mem_get_integer(left, &l, &unused) != 0) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 			 mem_str(left), "integer");
 		return -1;
 	}
-	if (sqlVdbeIntValue(right, &r, &unused) != 0) {
+	if (mem_get_integer(right, &r, &unused) != 0) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 			 mem_str(right), "integer");
 		return -1;
@@ -1586,7 +1618,7 @@ mem_bit_not(struct Mem *mem, struct Mem *result)
 		return 0;
 	int64_t i;
 	bool unused;
-	if (sqlVdbeIntValue(mem, &i, &unused) != 0) {
+	if (mem_get_integer(mem, &i, &unused) != 0) {
 		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
 			 mem_str(mem), "integer");
 		return -1;
@@ -1867,35 +1899,6 @@ valueToText(sql_value * pVal)
 	return pVal->z;
 }
 
-/*
- * Convert a 64-bit IEEE double into a 64-bit signed integer.
- * If the double is out of range of a 64-bit signed integer then
- * return the closest available 64-bit signed integer.
- */
-static int
-doubleToInt64(double r, int64_t *i)
-{
-	/*
-	 * Many compilers we encounter do not define constants for the
-	 * minimum and maximum 64-bit integers, or they define them
-	 * inconsistently.  And many do not understand the "LL" notation.
-	 * So we define our own static constants here using nothing
-	 * larger than a 32-bit integer constant.
-	 */
-	static const int64_t maxInt = LARGEST_INT64;
-	static const int64_t minInt = SMALLEST_INT64;
-	if (r <= (double)minInt) {
-		*i = minInt;
-		return -1;
-	} else if (r >= (double)maxInt) {
-		*i = maxInt;
-		return -1;
-	} else {
-		*i = (int64_t) r;
-		return *i != r;
-	}
-}
-
 /*
  * It is already known that pMem contains an unterminated string.
  * Add the zero terminator.
@@ -2424,42 +2427,6 @@ mem_value_bool(const struct Mem *mem, bool *b)
 	return -1;
 }
 
-/*
- * Return some kind of integer value which is the best we can do
- * at representing the value that *pMem describes as an integer.
- * If pMem is an integer, then the value is exact.  If pMem is
- * a floating-point then the value returned is the integer part.
- * If pMem is a string or blob, then we make an attempt to convert
- * it into an integer and return that.  If pMem represents an
- * an SQL-NULL value, return 0.
- *
- * If pMem represents a string value, its encoding might be changed.
- */
-int
-sqlVdbeIntValue(Mem * pMem, int64_t *i, bool *is_neg)
-{
-	int flags;
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-	flags = pMem->flags;
-	if (flags & MEM_Int) {
-		*i = pMem->u.i;
-		*is_neg = true;
-		return 0;
-	} else if (flags & MEM_UInt) {
-		*i = pMem->u.u;
-		*is_neg = false;
-		return 0;
-	} else if (flags & MEM_Real) {
-		*is_neg = pMem->u.r < 0;
-		return doubleToInt64(pMem->u.r, i);
-	} else if (flags & (MEM_Str)) {
-		assert(pMem->z || pMem->n == 0);
-		if (sql_atoi64(pMem->z, i, is_neg, pMem->n) == 0)
-			return 0;
-	}
-	return -1;
-}
-
 /*
  * Return the best representation of pMem that we can get into a
  * double.  If pMem is already a double or an integer, return its
@@ -2530,30 +2497,12 @@ sql_value_boolean(sql_value *val)
 	return b;
 }
 
-int
-sql_value_int(sql_value * pVal)
-{
-	int64_t i = 0;
-	bool is_neg;
-	sqlVdbeIntValue((Mem *) pVal, &i, &is_neg);
-	return (int)i;
-}
-
-sql_int64
-sql_value_int64(sql_value * pVal)
-{
-	int64_t i = 0;
-	bool unused;
-	sqlVdbeIntValue((Mem *) pVal, &i, &unused);
-	return i;
-}
-
 uint64_t
 sql_value_uint64(sql_value *val)
 {
 	int64_t i = 0;
 	bool is_neg;
-	sqlVdbeIntValue((struct Mem *) val, &i, &is_neg);
+	mem_get_integer((struct Mem *) val, &i, &is_neg);
 	assert(!is_neg);
 	return i;
 }
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index c0b9f4f99..7857efc1d 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -345,6 +345,9 @@ mem_implicit_cast(struct Mem *mem, enum field_type type);
 int
 mem_implicit_cast_old(struct Mem *mem, enum field_type type);
 
+int
+mem_get_integer(const struct Mem *mem, int64_t *i, bool *is_neg);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -423,7 +426,6 @@ releaseMemArray(Mem * p, int N);
 
 int
 mem_value_bool(const struct Mem *mem, bool *b);
-int sqlVdbeIntValue(struct Mem *, int64_t *, bool *is_neg);
 int sqlVdbeRealValue(struct Mem *, double *);
 const void *
 sql_value_blob(struct Mem *);
@@ -437,12 +439,6 @@ sql_value_double(struct Mem *);
 bool
 sql_value_boolean(struct Mem *val);
 
-int
-sql_value_int(struct Mem *);
-
-sql_int64
-sql_value_int64(struct Mem *);
-
 uint64_t
 sql_value_uint64(struct Mem *val);
 
diff --git a/src/box/sql/printf.c b/src/box/sql/printf.c
index cf32ba3f3..d9140b484 100644
--- a/src/box/sql/printf.c
+++ b/src/box/sql/printf.c
@@ -144,7 +144,10 @@ getIntArg(PrintfArguments * p)
 {
 	if (p->nArg <= p->nUsed)
 		return 0;
-	return sql_value_int64(p->apArg[p->nUsed++]);
+	int64_t i;
+	bool unused;
+	mem_get_integer(p->apArg[p->nUsed++], &i, &unused);
+	return (sql_int64)i;
 }
 
 static double
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index f7ead1343..14385705f 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -445,15 +445,9 @@ sql_column_bytes16(sql_stmt *, int iCol);
 double
 sql_column_double(sql_stmt *, int iCol);
 
-int
-sql_column_int(sql_stmt *, int iCol);
-
 bool
 sql_column_boolean(struct sql_stmt *stmt, int column);
 
-sql_int64
-sql_column_int64(sql_stmt *, int iCol);
-
 uint64_t
 sql_column_uint64(struct sql_stmt *stmt, int column);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 8f871ea95..abdc5f0d0 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -491,24 +491,12 @@ sql_column_double(sql_stmt * pStmt, int i)
 	return sql_value_double(columnMem(pStmt, i));
 }
 
-int
-sql_column_int(sql_stmt * pStmt, int i)
-{
-	return sql_value_int(columnMem(pStmt, i));
-}
-
 bool
 sql_column_boolean(struct sql_stmt *stmt, int i)
 {
 	return sql_value_boolean(columnMem(stmt, i));
 }
 
-sql_int64
-sql_column_int64(sql_stmt * pStmt, int i)
-{
-	return sql_value_int64(columnMem(pStmt, i));
-}
-
 uint64_t
 sql_column_uint64(sql_stmt * pStmt, int i)
 {
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 48/53] sql: introduce mem_get_unsigned()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (46 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 47/53] sql: introduce mem_get_integer() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 49/53] sql: introduce mem_get_double() Mergen Imeev via Tarantool-patches
                   ` (4 subsequent siblings)
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_unsigned() function which is used to
receive unsigned value from MEM.

Part of #5818
---
 src/box/sql/func.c    | 16 +++++++++++-----
 src/box/sql/mem.c     | 37 +++++++++++++++++++++++++++----------
 src/box/sql/mem.h     |  6 +++---
 src/box/sql/sqlInt.h  |  3 ---
 src/box/sql/vdbeapi.c |  6 ------
 5 files changed, 41 insertions(+), 27 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 0fa0f6ac7..a851d98f2 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -118,9 +118,12 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 			luaL_pushint64(L, n);
 			break;
 		}
-		case MP_UINT:
-			luaL_pushuint64(L, sql_value_uint64(param));
+		case MP_UINT: {
+			uint64_t u;
+			mem_get_unsigned(param, &u);
+			luaL_pushuint64(L, u);
 			break;
+		}
 		case MP_DOUBLE:
 			lua_pushnumber(L, sql_value_double(param));
 			break;
@@ -171,7 +174,8 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 			FALLTHROUGH;
 		}
 		case MP_UINT: {
-			sql_uint64 val = sql_value_uint64(param);
+			uint64_t val;
+			mem_get_unsigned(param, &val);
 			mpstream_encode_uint(&stream, val);
 			break;
 		}
@@ -511,7 +515,9 @@ absFunc(sql_context * context, int argc, sql_value ** argv)
 	UNUSED_PARAMETER(argc);
 	switch (sql_value_type(argv[0])) {
 	case MP_UINT: {
-		sql_result_uint(context, sql_value_uint64(argv[0]));
+		uint64_t u;
+		mem_get_unsigned(argv[0], &u);
+		sql_result_uint(context, u);
 		break;
 	}
 	case MP_INT: {
@@ -1487,7 +1493,7 @@ charFunc(sql_context * context, int argc, sql_value ** argv)
 		if (sql_value_type(argv[i]) == MP_INT)
 			x = 0xfffd;
 		else
-			x = sql_value_uint64(argv[i]);
+			mem_get_unsigned(argv[i], &x);
 		if (x > 0x10ffff)
 			x = 0xfffd;
 		c = (unsigned)(x & 0x1fffff);
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 540931723..6ab6d77ad 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1259,6 +1259,33 @@ mem_get_integer(const struct Mem *mem, int64_t *i, bool *is_neg)
 	return -1;
 }
 
+int
+mem_get_unsigned(const struct Mem *mem, uint64_t *u)
+{
+	if ((mem->flags & MEM_Int) != 0)
+		return -1;
+	if ((mem->flags & MEM_UInt) != 0) {
+		*u = mem->u.u;
+		return 0;
+	}
+	if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) {
+		bool is_neg;
+		if (sql_atoi64(mem->z, (int64_t *)u, &is_neg, mem->n) != 0 ||
+		    is_neg)
+			return -1;
+		return 0;
+	}
+	if ((mem->flags & MEM_Real) != 0) {
+		double d = mem->u.r;
+		if (d >= 0 && d < (double)UINT64_MAX) {
+			*u = (uint64_t)d;
+			return 0;
+		}
+		return -1;
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2497,16 +2524,6 @@ sql_value_boolean(sql_value *val)
 	return b;
 }
 
-uint64_t
-sql_value_uint64(sql_value *val)
-{
-	int64_t i = 0;
-	bool is_neg;
-	mem_get_integer((struct Mem *) val, &i, &is_neg);
-	assert(!is_neg);
-	return i;
-}
-
 const unsigned char *
 sql_value_text(sql_value * pVal)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 7857efc1d..ebc9c6c20 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -348,6 +348,9 @@ mem_implicit_cast_old(struct Mem *mem, enum field_type type);
 int
 mem_get_integer(const struct Mem *mem, int64_t *i, bool *is_neg);
 
+int
+mem_get_unsigned(const struct Mem *mem, uint64_t *u);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -439,9 +442,6 @@ sql_value_double(struct Mem *);
 bool
 sql_value_boolean(struct Mem *val);
 
-uint64_t
-sql_value_uint64(struct Mem *val);
-
 const unsigned char *
 sql_value_text(struct Mem *);
 
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 14385705f..7f9539aea 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -448,9 +448,6 @@ sql_column_double(sql_stmt *, int iCol);
 bool
 sql_column_boolean(struct sql_stmt *stmt, int column);
 
-uint64_t
-sql_column_uint64(struct sql_stmt *stmt, int column);
-
 const unsigned char *
 sql_column_text(sql_stmt *,
 		    int iCol);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index abdc5f0d0..b6db98031 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -497,12 +497,6 @@ sql_column_boolean(struct sql_stmt *stmt, int i)
 	return sql_value_boolean(columnMem(stmt, i));
 }
 
-uint64_t
-sql_column_uint64(sql_stmt * pStmt, int i)
-{
-	return sql_value_uint64(columnMem(pStmt, i));
-}
-
 const unsigned char *
 sql_column_text(sql_stmt * pStmt, int i)
 {
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 49/53] sql: introduce mem_get_double()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (47 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 48/53] sql: introduce mem_get_unsigned() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 50/53] sql: introduce mem_get_boolean() Mergen Imeev via Tarantool-patches
                   ` (3 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_double() which is used to receive double
value from MEM.

Part of #5818
---
 src/box/sql/func.c    | 29 +++++++++++++---------
 src/box/sql/mem.c     | 57 +++++++++++++++++--------------------------
 src/box/sql/mem.h     |  7 +++---
 src/box/sql/printf.c  |  4 ++-
 src/box/sql/sqlInt.h  |  3 ---
 src/box/sql/vdbeapi.c |  6 -----
 6 files changed, 47 insertions(+), 59 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index a851d98f2..15fb2f14f 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -124,9 +124,12 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 			luaL_pushuint64(L, u);
 			break;
 		}
-		case MP_DOUBLE:
-			lua_pushnumber(L, sql_value_double(param));
+		case MP_DOUBLE: {
+			double d;
+			mem_get_double(param, &d);
+			lua_pushnumber(L, d);
 			break;
+		}
 		case MP_STR:
 			lua_pushstring(L, (const char *) sql_value_text(param));
 			break;
@@ -180,8 +183,9 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 			break;
 		}
 		case MP_DOUBLE: {
-			mpstream_encode_double(&stream,
-					       sql_value_double(param));
+			double d;
+			mem_get_double(param, &d);
+			mpstream_encode_double(&stream, d);
 			break;
 		}
 		case MP_STR: {
@@ -543,12 +547,13 @@ absFunc(sql_context * context, int argc, sql_value ** argv)
 		return;
 	}
 	default:{
-			/* Because sql_value_double() returns 0.0 if the argument is not
-			 * something that can be converted into a number, we have:
-			 * IMP: R-01992-00519 Abs(X) returns 0.0 if X is a string or blob
+			/*
+			 * Abs(X) returns 0.0 if X is a string or blob
 			 * that cannot be converted to a numeric value.
 			 */
-			double rVal = sql_value_double(argv[0]);
+			double rVal;
+			if (mem_get_double(argv[0], &rVal) != 0)
+				rVal = 0;
 			if (rVal < 0)
 				rVal = -rVal;
 			sql_result_double(context, rVal);
@@ -866,7 +871,7 @@ roundFunc(sql_context * context, int argc, sql_value ** argv)
 		context->is_aborted = true;
 		return;
 	}
-	r = sql_value_double(argv[0]);
+	mem_get_double(argv[0], &r);
 	/* If Y==0 and X will fit in a 64-bit int,
 	 * handle the rounding directly,
 	 * otherwise use printf.
@@ -1373,7 +1378,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 	case MP_DOUBLE:{
 			double r1, r2;
 			char zBuf[50];
-			r1 = sql_value_double(argv[0]);
+			mem_get_double(argv[0], &r1);
 			sql_snprintf(sizeof(zBuf), zBuf, "%!.15g", r1);
 			sqlAtoF(zBuf, &r2, 20);
 			if (r1 != r2) {
@@ -1994,7 +1999,9 @@ sum_step(struct sql_context *context, int argc, sql_value **argv)
 			p->overflow = 1;
 		}
 	} else {
-		p->rSum += sql_value_double(argv[0]);
+		double d;
+		mem_get_double(argv[0], &d);
+		p->rSum += d;
 		p->approx = 1;
 	}
 }
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 6ab6d77ad..8d4f0f92c 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1286,6 +1286,29 @@ mem_get_unsigned(const struct Mem *mem, uint64_t *u)
 	return -1;
 }
 
+int
+mem_get_double(const struct Mem *mem, double *d)
+{
+	if ((mem->flags & MEM_Real) != 0) {
+		*d = mem->u.r;
+		return 0;
+	}
+	if ((mem->flags & MEM_Int) != 0) {
+		*d = (double)mem->u.i;
+		return 0;
+	}
+	if ((mem->flags & MEM_UInt) != 0) {
+		*d = (double)mem->u.u;
+		return 0;
+	}
+	if ((mem->flags & MEM_Str) != 0) {
+		if (sqlAtoF(mem->z, d, mem->n) == 0)
+			return -1;
+		return 0;
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2454,32 +2477,6 @@ mem_value_bool(const struct Mem *mem, bool *b)
 	return -1;
 }
 
-/*
- * Return the best representation of pMem that we can get into a
- * double.  If pMem is already a double or an integer, return its
- * value.  If it is a string or blob, try to convert it to a double.
- * If it is a NULL, return 0.0.
- */
-int
-sqlVdbeRealValue(Mem * pMem, double *v)
-{
-	assert(EIGHT_BYTE_ALIGNMENT(pMem));
-	if (pMem->flags & MEM_Real) {
-		*v = pMem->u.r;
-		return 0;
-	} else if (pMem->flags & MEM_Int) {
-		*v = (double)pMem->u.i;
-		return 0;
-	} else if ((pMem->flags & MEM_UInt) != 0) {
-		*v = (double)pMem->u.u;
-		return 0;
-	} else if (pMem->flags & MEM_Str) {
-		if (sqlAtoF(pMem->z, v, pMem->n))
-			return 0;
-	}
-	return -1;
-}
-
 /**************************** sql_value_  ******************************
  * The following routines extract information from a Mem or sql_value
  * structure.
@@ -2506,14 +2503,6 @@ sql_value_bytes(sql_value * pVal)
 	return sqlValueBytes(pVal);
 }
 
-double
-sql_value_double(sql_value * pVal)
-{
-	double v = 0.0;
-	sqlVdbeRealValue((Mem *) pVal, &v);
-	return v;
-}
-
 bool
 sql_value_boolean(sql_value *val)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index ebc9c6c20..f9d229baf 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -351,6 +351,9 @@ mem_get_integer(const struct Mem *mem, int64_t *i, bool *is_neg);
 int
 mem_get_unsigned(const struct Mem *mem, uint64_t *u);
 
+int
+mem_get_double(const struct Mem *mem, double *d);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -429,16 +432,12 @@ releaseMemArray(Mem * p, int N);
 
 int
 mem_value_bool(const struct Mem *mem, bool *b);
-int sqlVdbeRealValue(struct Mem *, double *);
 const void *
 sql_value_blob(struct Mem *);
 
 int
 sql_value_bytes(struct Mem *);
 
-double
-sql_value_double(struct Mem *);
-
 bool
 sql_value_boolean(struct Mem *val);
 
diff --git a/src/box/sql/printf.c b/src/box/sql/printf.c
index d9140b484..0790a52a7 100644
--- a/src/box/sql/printf.c
+++ b/src/box/sql/printf.c
@@ -155,7 +155,9 @@ getDoubleArg(PrintfArguments * p)
 {
 	if (p->nArg <= p->nUsed)
 		return 0.0;
-	return sql_value_double(p->apArg[p->nUsed++]);
+	double d;
+	mem_get_double(p->apArg[p->nUsed++], &d);
+	return d;
 }
 
 static char *
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 7f9539aea..3e2cc4a66 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -442,9 +442,6 @@ sql_column_bytes(sql_stmt *, int iCol);
 int
 sql_column_bytes16(sql_stmt *, int iCol);
 
-double
-sql_column_double(sql_stmt *, int iCol);
-
 bool
 sql_column_boolean(struct sql_stmt *stmt, int column);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index b6db98031..9a97b2d77 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -485,12 +485,6 @@ sql_column_bytes(sql_stmt * pStmt, int i)
 	return sql_value_bytes(columnMem(pStmt, i));
 }
 
-double
-sql_column_double(sql_stmt * pStmt, int i)
-{
-	return sql_value_double(columnMem(pStmt, i));
-}
-
 bool
 sql_column_boolean(struct sql_stmt *stmt, int i)
 {
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 50/53] sql: introduce mem_get_boolean()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (48 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 49/53] sql: introduce mem_get_double() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 51/53] sql: introduce mem_get_string0() Mergen Imeev via Tarantool-patches
                   ` (2 subsequent siblings)
  52 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_boolean() which is used to receive boolean
value from MEM.

Part of #5818
---
 src/box/sql/func.c    | 16 +++++++++++-----
 src/box/sql/mem.c     | 30 ++++++++++--------------------
 src/box/sql/mem.h     |  8 +++-----
 src/box/sql/sqlInt.h  |  3 ---
 src/box/sql/vdbeapi.c |  6 ------
 5 files changed, 24 insertions(+), 39 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 15fb2f14f..d033dae86 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -142,9 +142,12 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 		case MP_NIL:
 			lua_pushnil(L);
 			break;
-		case MP_BOOL:
-			lua_pushboolean(L, sql_value_boolean(param));
+		case MP_BOOL: {
+			bool b;
+			mem_get_boolean(param, &b);
+			lua_pushboolean(L, b);
 			break;
+		}
 		default:
 			unreachable();
 		}
@@ -206,7 +209,9 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 			break;
 		}
 		case MP_BOOL: {
-			mpstream_encode_bool(&stream, sql_value_boolean(param));
+			bool b;
+			mem_get_boolean(param, &b);
+			mpstream_encode_bool(&stream, b);
 			break;
 		}
 		default:
@@ -1451,8 +1456,9 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 			break;
 		}
 	case MP_BOOL: {
-		sql_result_text(context,
-				SQL_TOKEN_BOOLEAN(sql_value_boolean(argv[0])),
+		bool b;
+		mem_get_boolean(argv[0], &b);
+		sql_result_text(context, SQL_TOKEN_BOOLEAN(b),
 				-1, SQL_TRANSIENT);
 		break;
 	}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 8d4f0f92c..90c6ee526 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1309,6 +1309,16 @@ mem_get_double(const struct Mem *mem, double *d)
 	return -1;
 }
 
+int
+mem_get_boolean(const struct Mem *mem, bool *b)
+{
+	if ((mem->flags & MEM_Bool) != 0) {
+		*b = mem->u.b;
+		return 0;
+	}
+	return -1;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2467,16 +2477,6 @@ releaseMemArray(Mem * p, int N)
 	}
 }
 
-int
-mem_value_bool(const struct Mem *mem, bool *b)
-{
-	if ((mem->flags  & MEM_Bool) != 0) {
-		*b = mem->u.b;
-		return 0;
-	}
-	return -1;
-}
-
 /**************************** sql_value_  ******************************
  * The following routines extract information from a Mem or sql_value
  * structure.
@@ -2503,16 +2503,6 @@ sql_value_bytes(sql_value * pVal)
 	return sqlValueBytes(pVal);
 }
 
-bool
-sql_value_boolean(sql_value *val)
-{
-	bool b = false;
-	int rc = mem_value_bool((struct Mem *) val, &b);
-	assert(rc == 0);
-	(void) rc;
-	return b;
-}
-
 const unsigned char *
 sql_value_text(sql_value * pVal)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index f9d229baf..8015608fa 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -354,6 +354,9 @@ mem_get_unsigned(const struct Mem *mem, uint64_t *u);
 int
 mem_get_double(const struct Mem *mem, double *d);
 
+int
+mem_get_boolean(const struct Mem *mem, bool *b);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -430,17 +433,12 @@ releaseMemArray(Mem * p, int N);
 
 /** Getters. */
 
-int
-mem_value_bool(const struct Mem *mem, bool *b);
 const void *
 sql_value_blob(struct Mem *);
 
 int
 sql_value_bytes(struct Mem *);
 
-bool
-sql_value_boolean(struct Mem *val);
-
 const unsigned char *
 sql_value_text(struct Mem *);
 
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 3e2cc4a66..9b4afc3f2 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -442,9 +442,6 @@ sql_column_bytes(sql_stmt *, int iCol);
 int
 sql_column_bytes16(sql_stmt *, int iCol);
 
-bool
-sql_column_boolean(struct sql_stmt *stmt, int column);
-
 const unsigned char *
 sql_column_text(sql_stmt *,
 		    int iCol);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 9a97b2d77..532677b05 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -485,12 +485,6 @@ sql_column_bytes(sql_stmt * pStmt, int i)
 	return sql_value_bytes(columnMem(pStmt, i));
 }
 
-bool
-sql_column_boolean(struct sql_stmt *stmt, int i)
-{
-	return sql_value_boolean(columnMem(stmt, i));
-}
-
 const unsigned char *
 sql_column_text(sql_stmt * pStmt, int i)
 {
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 51/53] sql: introduce mem_get_string0()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (49 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 50/53] sql: introduce mem_get_boolean() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 52/53] sql: introduce mem_get_binary() Mergen Imeev via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 53/53] sql: introduce mem_get_length() Mergen Imeev via Tarantool-patches
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_string0() which is used to receive
NULL-terminated string from MEM.

Part of #5818
---
 src/box/sql/func.c      |  77 ++++++++++++++++++------------
 src/box/sql/mem.c       | 101 ++++++----------------------------------
 src/box/sql/mem.h       |   9 ++--
 src/box/sql/printf.c    |   6 ++-
 src/box/sql/sqlInt.h    |   4 --
 src/box/sql/vdbeapi.c   |   6 ---
 src/box/sql/whereexpr.c |   7 ++-
 7 files changed, 75 insertions(+), 135 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index d033dae86..78f4ec3b5 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -54,6 +54,24 @@
 #include "lua/utils.h"
 #include "mpstream/mpstream.h"
 
+static const char *
+mem_get_str(struct Mem *mem)
+{
+	const char *str;
+	if (mem_convert_to_string0(mem) != 0 || mem_get_string0(mem, &str) != 0)
+		return NULL;
+	return str;
+}
+
+static const unsigned char *
+mem_get_ustr(struct Mem *mem)
+{
+	const char *str;
+	if (mem_convert_to_string0(mem) != 0 || mem_get_string0(mem, &str) != 0)
+		return NULL;
+	return (const unsigned char *)str;
+}
+
 /*
  * Return the collating function associated with a function.
  */
@@ -131,7 +149,7 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 			break;
 		}
 		case MP_STR:
-			lua_pushstring(L, (const char *) sql_value_text(param));
+			lua_pushstring(L, mem_get_str(param));
 			break;
 		case MP_BIN:
 		case MP_ARRAY:
@@ -192,7 +210,7 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 			break;
 		}
 		case MP_STR: {
-			const char *str = (const char *) sql_value_text(param);
+			const char *str = mem_get_str(param);
 			mpstream_encode_str(&stream, str);
 			break;
 		}
@@ -497,7 +515,7 @@ lengthFunc(sql_context * context, int argc, sql_value ** argv)
 			break;
 		}
 	case MP_STR:{
-			const unsigned char *z = sql_value_text(argv[0]);
+			const unsigned char *z = mem_get_ustr(argv[0]);
 			if (z == 0)
 				return;
 			len = sql_utf8_char_count(z, sql_value_bytes(argv[0]));
@@ -652,8 +670,8 @@ position_func(struct sql_context *context, int argc, struct Mem **argv)
 			 * Character size is equal to
 			 * needle char size.
 			 */
-			haystack_str = sql_value_text(haystack);
-			needle_str = sql_value_text(needle);
+			haystack_str = mem_get_ustr(haystack);
+			needle_str = mem_get_ustr(needle);
 
 			int n_needle_chars =
 				sql_utf8_char_count(needle_str, n_needle_bytes);
@@ -715,8 +733,7 @@ printfFunc(sql_context * context, int argc, sql_value ** argv)
 	int n;
 	sql *db = sql_context_db_handle(context);
 
-	if (argc >= 1
-	    && (zFormat = (const char *)sql_value_text(argv[0])) != 0) {
+	if (argc >= 1 && (zFormat = mem_get_str(argv[0])) != NULL) {
 		x.nArg = argc - 1;
 		x.nUsed = 0;
 		x.apArg = argv + 1;
@@ -773,7 +790,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 			return;
 		assert(len == sql_value_bytes(argv[0]));
 	} else {
-		z = sql_value_text(argv[0]);
+		z = mem_get_ustr(argv[0]);
 		if (z == 0)
 			return;
 		len = 0;
@@ -936,13 +953,13 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv)   \
 		context->is_aborted = true;                                    \
 		return;                                                        \
 	}                                                                      \
-	z2 = (char *)sql_value_text(argv[0]);                              \
+	z2 = mem_get_str(argv[0]);                                            \
 	n = sql_value_bytes(argv[0]);                                      \
 	/*                                                                     \
 	 * Verify that the call to _bytes()                                    \
 	 * does not invalidate the _text() pointer.                            \
 	 */                                                                    \
-	assert(z2 == (char *)sql_value_text(argv[0]));                     \
+	assert(z2 == mem_get_str(argv[0]));                                   \
 	if (!z2)                                                               \
 		return;                                                        \
 	z1 = contextMalloc(context, ((i64) n) + 1);                            \
@@ -1273,8 +1290,8 @@ likeFunc(sql_context *context, int argc, sql_value **argv)
 		context->is_aborted = true;
 		return;
 	}
-	const char *zB = (const char *) sql_value_text(argv[0]);
-	const char *zA = (const char *) sql_value_text(argv[1]);
+	const char *zB = mem_get_str(argv[0]);
+	const char *zA = mem_get_str(argv[1]);
 	const char *zB_end = zB + sql_value_bytes(argv[0]);
 	const char *zA_end = zA + sql_value_bytes(argv[1]);
 
@@ -1293,7 +1310,7 @@ likeFunc(sql_context *context, int argc, sql_value **argv)
 		return;
 	}
 	/* Encoding did not change */
-	assert(zB == (const char *) sql_value_text(argv[0]));
+	assert(zB == mem_get_str(argv[0]));
 
 	if (argc == 3) {
 		/*
@@ -1301,7 +1318,7 @@ likeFunc(sql_context *context, int argc, sql_value **argv)
 		 * single UTF-8 character. Otherwise, return an
 		 * error.
 		 */
-		const unsigned char *zEsc = sql_value_text(argv[2]);
+		const unsigned char *zEsc = mem_get_ustr(argv[2]);
 		if (zEsc == 0)
 			return;
 		if (sql_utf8_char_count(zEsc, sql_value_bytes(argv[2])) != 1) {
@@ -1430,7 +1447,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 	case MP_STR:{
 			int i, j;
 			u64 n;
-			const unsigned char *zArg = sql_value_text(argv[0]);
+			const unsigned char *zArg = mem_get_ustr(argv[0]);
 			char *z;
 
 			if (zArg == 0)
@@ -1477,7 +1494,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 static void
 unicodeFunc(sql_context * context, int argc, sql_value ** argv)
 {
-	const unsigned char *z = sql_value_text(argv[0]);
+	const unsigned char *z = mem_get_ustr(argv[0]);
 	(void)argc;
 	if (z && z[0])
 		sql_result_uint(context, sqlUtf8Read(&z));
@@ -1596,12 +1613,12 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
 
 	assert(argc == 3);
 	UNUSED_PARAMETER(argc);
-	zStr = sql_value_text(argv[0]);
+	zStr = mem_get_ustr(argv[0]);
 	if (zStr == 0)
 		return;
 	nStr = sql_value_bytes(argv[0]);
-	assert(zStr == sql_value_text(argv[0]));	/* No encoding change */
-	zPattern = sql_value_text(argv[1]);
+	assert(zStr == mem_get_ustr(argv[0]));	/* No encoding change */
+	zPattern = mem_get_ustr(argv[1]);
 	if (zPattern == 0) {
 		assert(mem_is_null(argv[1])
 		       || sql_context_db_handle(context)->mallocFailed);
@@ -1613,12 +1630,12 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
 		sql_result_value(context, argv[0]);
 		return;
 	}
-	assert(zPattern == sql_value_text(argv[1]));	/* No encoding change */
-	zRep = sql_value_text(argv[2]);
+	assert(zPattern == mem_get_ustr(argv[1]));	/* No encoding change */
+	zRep = mem_get_ustr(argv[2]);
 	if (zRep == 0)
 		return;
 	nRep = sql_value_bytes(argv[2]);
-	assert(zRep == sql_value_text(argv[2]));
+	assert(zRep == mem_get_ustr(argv[2]));
 	nOut = nStr + 1;
 	assert(nOut < SQL_MAX_LENGTH);
 	zOut = contextMalloc(context, (i64) nOut);
@@ -1780,7 +1797,7 @@ trim_func_one_arg(struct sql_context *context, sql_value *arg)
 	else
 		default_trim = (const unsigned char *) " ";
 	int input_str_sz = sql_value_bytes(arg);
-	const unsigned char *input_str = sql_value_text(arg);
+	const unsigned char *input_str = mem_get_ustr(arg);
 	uint8_t trim_char_len[1] = { 1 };
 	trim_procedure(context, TRIM_BOTH, default_trim, trim_char_len, 1,
 		       input_str, input_str_sz);
@@ -1802,7 +1819,7 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1,
 		   sql_value *arg2)
 {
 	const unsigned char *input_str, *trim_set;
-	if ((input_str = sql_value_text(arg2)) == NULL)
+	if ((input_str = mem_get_ustr(arg2)) == NULL)
 		return;
 
 	int input_str_sz = sql_value_bytes(arg2);
@@ -1813,7 +1830,7 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1,
 		mem_get_integer(arg1, &n, &unused);
 		trim_procedure(context, n, (const unsigned char *) " ",
 			       &len_one, 1, input_str, input_str_sz);
-	} else if ((trim_set = sql_value_text(arg1)) != NULL) {
+	} else if ((trim_set = mem_get_ustr(arg1)) != NULL) {
 		int trim_set_sz = sql_value_bytes(arg1);
 		uint8_t *char_len;
 		int char_cnt = trim_prepare_char_len(context, trim_set,
@@ -1839,8 +1856,8 @@ trim_func_three_args(struct sql_context *context, sql_value *arg1,
 {
 	assert(sql_value_type(arg1) == MP_INT || sql_value_type(arg1) == MP_UINT);
 	const unsigned char *input_str, *trim_set;
-	if ((input_str = sql_value_text(arg3)) == NULL ||
-	    (trim_set = sql_value_text(arg2)) == NULL)
+	if ((input_str = mem_get_ustr(arg3)) == NULL ||
+	    (trim_set = mem_get_ustr(arg2)) == NULL)
 		return;
 
 	int trim_set_sz = sql_value_bytes(arg2);
@@ -1915,7 +1932,7 @@ soundexFunc(sql_context * context, int argc, sql_value ** argv)
 		context->is_aborted = true;
 		return;
 	}
-	zIn = (u8 *) sql_value_text(argv[0]);
+	zIn = (u8 *) mem_get_ustr(argv[0]);
 	if (zIn == 0)
 		zIn = (u8 *) "";
 	for (i = 0; zIn[i] && !sqlIsalpha(zIn[i]); i++) {
@@ -2165,7 +2182,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
 		pAccum->mxAlloc = db->aLimit[SQL_LIMIT_LENGTH];
 		if (!firstTerm) {
 			if (argc == 2) {
-				zSep = (char *)sql_value_text(argv[1]);
+				zSep = mem_get_str(argv[1]);
 				nSep = sql_value_bytes(argv[1]);
 			} else {
 				zSep = ",";
@@ -2174,7 +2191,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
 			if (zSep)
 				sqlStrAccumAppend(pAccum, zSep, nSep);
 		}
-		zVal = (char *)sql_value_text(argv[0]);
+		zVal = mem_get_str(argv[0]);
 		nVal = sql_value_bytes(argv[0]);
 		if (zVal)
 			sqlStrAccumAppend(pAccum, zVal, nVal);
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index 90c6ee526..efd3e4677 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1319,6 +1319,15 @@ mem_get_boolean(const struct Mem *mem, bool *b)
 	return -1;
 }
 
+int
+mem_get_string0(const struct Mem *mem, const char **s)
+{
+	if ((mem->flags & MEM_Str) == 0 || (mem->flags & MEM_Term) == 0)
+		return -1;
+	*s = mem->z;
+	return 0;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -1936,45 +1945,6 @@ mem_has_msgpack_subtype(struct Mem *mem)
 	       mem->subtype == SQL_SUBTYPE_MSGPACK;
 }
 
-/*
- * The pVal argument is known to be a value other than NULL.
- * Convert it into a string with encoding enc and return a pointer
- * to a zero-terminated version of that string.
- */
-static SQL_NOINLINE const void *
-valueToText(sql_value * pVal)
-{
-	assert(pVal != 0);
-	assert((pVal->flags & (MEM_Null)) == 0);
-	if ((pVal->flags & (MEM_Blob | MEM_Str)) &&
-	    !mem_has_msgpack_subtype(pVal)) {
-		if (ExpandBlob(pVal))
-			return 0;
-		pVal->flags |= MEM_Str;
-		sqlVdbeMemNulTerminate(pVal);	/* IMP: R-31275-44060 */
-	} else {
-		mem_convert_to_string(pVal);
-		assert(0 == (1 & SQL_PTR_TO_INT(pVal->z)));
-	}
-	return pVal->z;
-}
-
-/*
- * It is already known that pMem contains an unterminated string.
- * Add the zero terminator.
- */
-static SQL_NOINLINE int
-vdbeMemAddTerminator(Mem * pMem)
-{
-	if (sqlVdbeMemGrow(pMem, pMem->n + 2, 1)) {
-		return -1;
-	}
-	pMem->z[pMem->n] = 0;
-	pMem->z[pMem->n + 1] = 0;
-	pMem->flags |= MEM_Term;
-	return 0;
-}
-
 /*
  * Both *pMem1 and *pMem2 contain string values. Compare the two values
  * using the collation sequence pColl. As usual, return a negative , zero
@@ -2076,7 +2046,9 @@ sql_value_type(sql_value *pVal)
 static SQL_NOINLINE int
 valueBytes(sql_value * pVal)
 {
-	return valueToText(pVal) != 0 ? pVal->n : 0;
+	if (mem_convert_to_string(pVal) != 0)
+		return 0;
+	return pVal->n;
 }
 
 int
@@ -2311,21 +2283,6 @@ registerTrace(int iReg, Mem *p) {
 }
 #endif
 
-/*
- * Make sure the given Mem is \u0000 terminated.
- */
-int
-sqlVdbeMemNulTerminate(Mem * pMem)
-{
-	testcase((pMem->flags & (MEM_Term | MEM_Str)) == (MEM_Term | MEM_Str));
-	testcase((pMem->flags & (MEM_Term | MEM_Str)) == 0);
-	if ((pMem->flags & (MEM_Term | MEM_Str)) != MEM_Str) {
-		return 0;	/* Nothing to do */
-	} else {
-		return vdbeMemAddTerminator(pMem);
-	}
-}
-
 /*
  * If the given Mem* has a zero-filled tail, turn it into an ordinary
  * blob stored in dynamically allocated space.
@@ -2493,7 +2450,9 @@ sql_value_blob(sql_value * pVal)
 		p->flags |= MEM_Blob;
 		return p->n ? p->z : 0;
 	} else {
-		return sql_value_text(pVal);
+		if (mem_convert_to_string(pVal) != 0)
+			return NULL;
+		return pVal->z;
 	}
 }
 
@@ -2503,36 +2462,6 @@ sql_value_bytes(sql_value * pVal)
 	return sqlValueBytes(pVal);
 }
 
-const unsigned char *
-sql_value_text(sql_value * pVal)
-{
-	return (const unsigned char *)sqlValueText(pVal);
-}
-
-/* This function is only available internally, it is not part of the
- * external API. It works in a similar way to sql_value_text(),
- * except the data returned is in the encoding specified by the second
- * parameter, which must be one of SQL_UTF16BE, SQL_UTF16LE or
- * SQL_UTF8.
- *
- * (2006-02-16:)  The enc value can be or-ed with SQL_UTF16_ALIGNED.
- * If that is the case, then the result must be aligned on an even byte
- * boundary.
- */
-const void *
-sqlValueText(sql_value * pVal)
-{
-	if (!pVal)
-		return 0;
-	if ((pVal->flags & (MEM_Str | MEM_Term)) == (MEM_Str | MEM_Term)) {
-		return pVal->z;
-	}
-	if (pVal->flags & MEM_Null) {
-		return 0;
-	}
-	return valueToText(pVal);
-}
-
 /*
  * Return a pointer to static memory containing an SQL NULL value.
  */
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8015608fa..fb2385541 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -357,6 +357,9 @@ mem_get_double(const struct Mem *mem, double *d);
 int
 mem_get_boolean(const struct Mem *mem, bool *b);
 
+int
+mem_get_string0(const struct Mem *mem, const char **s);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -407,7 +410,6 @@ registerTrace(int iReg, Mem *p);
 # define memAboutToChange(P,M)
 #endif
 
-int sqlVdbeMemNulTerminate(struct Mem *);
 int sqlVdbeMemExpandBlob(struct Mem *);
 #define ExpandBlob(P) (mem_is_zeroblob(P)? sqlVdbeMemExpandBlob(P) : 0)
 
@@ -439,11 +441,6 @@ sql_value_blob(struct Mem *);
 int
 sql_value_bytes(struct Mem *);
 
-const unsigned char *
-sql_value_text(struct Mem *);
-
-const void *sqlValueText(struct Mem *);
-
 #define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
 
 const Mem *
diff --git a/src/box/sql/printf.c b/src/box/sql/printf.c
index 0790a52a7..85cfe8802 100644
--- a/src/box/sql/printf.c
+++ b/src/box/sql/printf.c
@@ -165,7 +165,11 @@ getTextArg(PrintfArguments * p)
 {
 	if (p->nArg <= p->nUsed)
 		return 0;
-	return (char *)sql_value_text(p->apArg[p->nUsed++]);
+	const char *str;
+	struct Mem *mem = p->apArg[p->nUsed++];
+	if (mem_convert_to_string0(mem) != 0 || mem_get_string0(mem, &str) != 0)
+		return NULL;
+	return (char *)str;
 }
 
 /*
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 9b4afc3f2..2f995ffc6 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -442,10 +442,6 @@ sql_column_bytes(sql_stmt *, int iCol);
 int
 sql_column_bytes16(sql_stmt *, int iCol);
 
-const unsigned char *
-sql_column_text(sql_stmt *,
-		    int iCol);
-
 char *
 sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
 		      struct region *region);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 532677b05..465284567 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -485,12 +485,6 @@ sql_column_bytes(sql_stmt * pStmt, int i)
 	return sql_value_bytes(columnMem(pStmt, i));
 }
 
-const unsigned char *
-sql_column_text(sql_stmt * pStmt, int i)
-{
-	return sql_value_text(columnMem(pStmt, i));
-}
-
 char *
 sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
 		      struct region *region)
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 782e44734..51681b008 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -312,8 +312,11 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
 		pVal =
 		    sqlVdbeGetBoundValue(pReprepare, iCol,
 					     FIELD_TYPE_SCALAR);
-		if (pVal != NULL && mem_is_string(pVal))
-			z = (char *)sql_value_text(pVal);
+		if (pVal != NULL && mem_is_string(pVal)) {
+			if (mem_convert_to_string0(pVal) != 0 ||
+			    mem_get_string0(pVal, &z) != 0)
+				return -1;
+		}
 		assert(pRight->op == TK_VARIABLE || pRight->op == TK_REGISTER);
 	} else if (op == TK_STRING) {
 		z = pRight->u.zToken;
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 52/53] sql: introduce mem_get_binary()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (50 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 51/53] sql: introduce mem_get_string0() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:09   ` Vladislav Shpilevoy via Tarantool-patches
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 53/53] sql: introduce mem_get_length() Mergen Imeev via Tarantool-patches
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_binary() which is used to receive binary
value from MEM.

Part of #5818
---
 src/box/sql/func.c    | 31 ++++++++++++++++++++++---------
 src/box/sql/mem.c     | 35 +++++++++++++----------------------
 src/box/sql/mem.h     |  6 +++---
 src/box/sql/sqlInt.h  |  3 ---
 src/box/sql/vdbeapi.c |  7 -------
 5 files changed, 38 insertions(+), 44 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 78f4ec3b5..199f3abef 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -72,6 +72,19 @@ mem_get_ustr(struct Mem *mem)
 	return (const unsigned char *)str;
 }
 
+static const void *
+mem_get_blob(struct Mem *mem)
+{
+	const char *s;
+	if (!mem_is_varstring(mem) && mem_convert_to_string(mem) != 0)
+		return NULL;
+	if (ExpandBlob(mem) != 0)
+		return NULL;
+	if (mem_get_binary(mem, &s) != 0)
+		return NULL;
+	return (const void *)s;
+}
+
 /*
  * Return the collating function associated with a function.
  */
@@ -154,7 +167,7 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 		case MP_BIN:
 		case MP_ARRAY:
 		case MP_MAP:
-			lua_pushlstring(L, sql_value_blob(param),
+			lua_pushlstring(L, mem_get_blob(param),
 					(size_t) sql_value_bytes(param));
 			break;
 		case MP_NIL:
@@ -218,7 +231,7 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 		case MP_ARRAY:
 		case MP_MAP: {
 			mpstream_encode_binl(&stream, sql_value_bytes(param));
-			mpstream_memcpy(&stream, sql_value_blob(param),
+			mpstream_memcpy(&stream, mem_get_blob(param),
 					sql_value_bytes(param));
 			break;
 		}
@@ -641,8 +654,8 @@ position_func(struct sql_context *context, int argc, struct Mem **argv)
 		const unsigned char *haystack_str;
 		const unsigned char *needle_str;
 		if (haystack_type == MP_BIN) {
-			needle_str = sql_value_blob(needle);
-			haystack_str = sql_value_blob(haystack);
+			needle_str = mem_get_blob(needle);
+			haystack_str = mem_get_blob(haystack);
 			assert(needle_str != NULL);
 			assert(haystack_str != NULL || n_haystack_bytes == 0);
 			/*
@@ -785,7 +798,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 	mem_get_integer(argv[1], &p1, &unused);
 	if (p0type == MP_BIN) {
 		len = sql_value_bytes(argv[0]);
-		z = sql_value_blob(argv[0]);
+		z = mem_get_blob(argv[0]);
 		if (z == 0)
 			return;
 		assert(len == sql_value_bytes(argv[0]));
@@ -1420,9 +1433,9 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 	case MP_ARRAY:
 	case MP_MAP: {
 			char *zText = 0;
-			char const *zBlob = sql_value_blob(argv[0]);
+			char const *zBlob = mem_get_blob(argv[0]);
 			int nBlob = sql_value_bytes(argv[0]);
-			assert(zBlob == sql_value_blob(argv[0]));	/* No encoding change */
+			assert(zBlob == mem_get_blob(argv[0]));	/* No encoding change */
 			zText =
 			    (char *)contextMalloc(context,
 						  (2 * (i64) nBlob) + 4);
@@ -1556,9 +1569,9 @@ hexFunc(sql_context * context, int argc, sql_value ** argv)
 	char *zHex, *z;
 	assert(argc == 1);
 	UNUSED_PARAMETER(argc);
-	pBlob = sql_value_blob(argv[0]);
+	pBlob = mem_get_blob(argv[0]);
 	n = sql_value_bytes(argv[0]);
-	assert(pBlob == sql_value_blob(argv[0]));	/* No encoding change */
+	assert(pBlob == mem_get_blob(argv[0]));	/* No encoding change */
 	z = zHex = contextMalloc(context, ((i64) n) * 2 + 1);
 	if (zHex) {
 		for (i = 0; i < n; i++, pBlob++) {
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index efd3e4677..bed9ae64a 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1328,6 +1328,19 @@ mem_get_string0(const struct Mem *mem, const char **s)
 	return 0;
 }
 
+int
+mem_get_binary(const struct Mem *mem, const char **s)
+{
+	if ((mem->flags & MEM_Str) != 0) {
+		*s = mem->n > 0 ? mem->z : NULL;
+		return 0;
+	}
+	if ((mem->flags & MEM_Blob) == 0 || (mem->flags & MEM_Zero) != 0)
+		return -1;
+	*s = mem->z;
+	return 0;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2434,28 +2447,6 @@ releaseMemArray(Mem * p, int N)
 	}
 }
 
-/**************************** sql_value_  ******************************
- * The following routines extract information from a Mem or sql_value
- * structure.
- */
-const void *
-sql_value_blob(sql_value * pVal)
-{
-	Mem *p = (Mem *) pVal;
-	if (p->flags & (MEM_Blob | MEM_Str)) {
-		if (ExpandBlob(p) != 0) {
-			assert(p->flags == MEM_Null && p->z == 0);
-			return 0;
-		}
-		p->flags |= MEM_Blob;
-		return p->n ? p->z : 0;
-	} else {
-		if (mem_convert_to_string(pVal) != 0)
-			return NULL;
-		return pVal->z;
-	}
-}
-
 int
 sql_value_bytes(sql_value * pVal)
 {
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index fb2385541..8af924d43 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -360,6 +360,9 @@ mem_get_boolean(const struct Mem *mem, bool *b);
 int
 mem_get_string0(const struct Mem *mem, const char **s);
 
+int
+mem_get_binary(const struct Mem *mem, const char **s);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -435,9 +438,6 @@ releaseMemArray(Mem * p, int N);
 
 /** Getters. */
 
-const void *
-sql_value_blob(struct Mem *);
-
 int
 sql_value_bytes(struct Mem *);
 
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 2f995ffc6..9dfd0e4cc 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -433,9 +433,6 @@ sql_stmt_compile(const char *sql, int bytes_count, struct Vdbe *re_prepared,
 int
 sql_step(sql_stmt *);
 
-const void *
-sql_column_blob(sql_stmt *, int iCol);
-
 int
 sql_column_bytes(sql_stmt *, int iCol);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 465284567..e76170965 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -471,13 +471,6 @@ columnMem(sql_stmt * pStmt, int i)
  * The following routines are used to access elements of the current row
  * in the result set.
  */
-const void *
-sql_column_blob(sql_stmt * pStmt, int i)
-{
-	const void *val;
-	val = sql_value_blob(columnMem(pStmt, i));
-	return val;
-}
 
 int
 sql_column_bytes(sql_stmt * pStmt, int i)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* [Tarantool-patches] [PATCH v4 53/53] sql: introduce mem_get_length()
  2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
                   ` (51 preceding siblings ...)
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 52/53] sql: introduce mem_get_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-23  9:36 ` Mergen Imeev via Tarantool-patches
  2021-03-29 23:09   ` Vladislav Shpilevoy via Tarantool-patches
  52 siblings, 1 reply; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-03-23  9:36 UTC (permalink / raw)
  To: v.shpilevoy, tsafin; +Cc: tarantool-patches

This patch introduces mem_get_length() which is used to receive length
of string or binary value of MEM.

Part of #5818
---
 src/box/sql/func.c    | 57 ++++++++++++++-------------
 src/box/sql/mem.c     | 92 +++++--------------------------------------
 src/box/sql/mem.h     | 15 ++-----
 src/box/sql/sqlInt.h  |  3 --
 src/box/sql/vdbeapi.c | 35 ----------------
 5 files changed, 42 insertions(+), 160 deletions(-)

diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 199f3abef..6687fca3c 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -168,7 +168,7 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
 		case MP_ARRAY:
 		case MP_MAP:
 			lua_pushlstring(L, mem_get_blob(param),
-					(size_t) sql_value_bytes(param));
+					(size_t) mem_get_length(param));
 			break;
 		case MP_NIL:
 			lua_pushnil(L);
@@ -230,9 +230,9 @@ port_vdbemem_get_msgpack(struct port *base, uint32_t *size)
 		case MP_BIN:
 		case MP_ARRAY:
 		case MP_MAP: {
-			mpstream_encode_binl(&stream, sql_value_bytes(param));
+			mpstream_encode_binl(&stream, mem_get_length(param));
 			mpstream_memcpy(&stream, mem_get_blob(param),
-					sql_value_bytes(param));
+					mem_get_length(param));
 			break;
 		}
 		case MP_NIL: {
@@ -524,14 +524,15 @@ lengthFunc(sql_context * context, int argc, sql_value ** argv)
 	case MP_UINT:
 	case MP_BOOL:
 	case MP_DOUBLE:{
-			sql_result_uint(context, sql_value_bytes(argv[0]));
+			mem_get_blob(argv[0]);
+			sql_result_uint(context, mem_get_length(argv[0]));
 			break;
 		}
 	case MP_STR:{
 			const unsigned char *z = mem_get_ustr(argv[0]);
 			if (z == 0)
 				return;
-			len = sql_utf8_char_count(z, sql_value_bytes(argv[0]));
+			len = sql_utf8_char_count(z, mem_get_length(argv[0]));
 			sql_result_uint(context, len);
 			break;
 		}
@@ -647,8 +648,8 @@ position_func(struct sql_context *context, int argc, struct Mem **argv)
 		return;
 	}
 
-	int n_needle_bytes = sql_value_bytes(needle);
-	int n_haystack_bytes = sql_value_bytes(haystack);
+	int n_needle_bytes = mem_get_length(needle);
+	int n_haystack_bytes = mem_get_length(haystack);
 	int position = 1;
 	if (n_needle_bytes > 0) {
 		const unsigned char *haystack_str;
@@ -797,18 +798,18 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 	bool unused;
 	mem_get_integer(argv[1], &p1, &unused);
 	if (p0type == MP_BIN) {
-		len = sql_value_bytes(argv[0]);
+		len = mem_get_length(argv[0]);
 		z = mem_get_blob(argv[0]);
 		if (z == 0)
 			return;
-		assert(len == sql_value_bytes(argv[0]));
+		assert(len == mem_get_length(argv[0]));
 	} else {
 		z = mem_get_ustr(argv[0]);
 		if (z == 0)
 			return;
 		len = 0;
 		if (p1 < 0)
-			len = sql_utf8_char_count(z, sql_value_bytes(argv[0]));
+			len = sql_utf8_char_count(z, mem_get_length(argv[0]));
 	}
 	if (argc == 3) {
 		mem_get_integer(argv[2], &p2, &unused);
@@ -847,7 +848,7 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
 		 * used because '\0' is not supposed to be
 		 * end-of-string symbol.
 		 */
-		int byte_size = sql_value_bytes(argv[0]);
+		int byte_size = mem_get_length(argv[0]);
 		int n_chars = sql_utf8_char_count(z, byte_size);
 		int cnt = 0;
 		int i = 0;
@@ -967,7 +968,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv)   \
 		return;                                                        \
 	}                                                                      \
 	z2 = mem_get_str(argv[0]);                                            \
-	n = sql_value_bytes(argv[0]);                                      \
+	n = mem_get_length(argv[0]);                                      \
 	/*                                                                     \
 	 * Verify that the call to _bytes()                                    \
 	 * does not invalidate the _text() pointer.                            \
@@ -1305,15 +1306,15 @@ likeFunc(sql_context *context, int argc, sql_value **argv)
 	}
 	const char *zB = mem_get_str(argv[0]);
 	const char *zA = mem_get_str(argv[1]);
-	const char *zB_end = zB + sql_value_bytes(argv[0]);
-	const char *zA_end = zA + sql_value_bytes(argv[1]);
+	const char *zB_end = zB + mem_get_length(argv[0]);
+	const char *zA_end = zA + mem_get_length(argv[1]);
 
 	/*
 	 * Limit the length of the LIKE pattern to avoid problems
 	 * of deep recursion and N*N behavior in
 	 * sql_utf8_pattern_compare().
 	 */
-	nPat = sql_value_bytes(argv[0]);
+	nPat = mem_get_length(argv[0]);
 	testcase(nPat == db->aLimit[SQL_LIMIT_LIKE_PATTERN_LENGTH]);
 	testcase(nPat == db->aLimit[SQL_LIMIT_LIKE_PATTERN_LENGTH] + 1);
 	if (nPat > db->aLimit[SQL_LIMIT_LIKE_PATTERN_LENGTH]) {
@@ -1334,7 +1335,7 @@ likeFunc(sql_context *context, int argc, sql_value **argv)
 		const unsigned char *zEsc = mem_get_ustr(argv[2]);
 		if (zEsc == 0)
 			return;
-		if (sql_utf8_char_count(zEsc, sql_value_bytes(argv[2])) != 1) {
+		if (sql_utf8_char_count(zEsc, mem_get_length(argv[2])) != 1) {
 			diag_set(ClientError, ER_SQL_EXECUTE, "ESCAPE "\
 				 "expression must be a single character");
 			context->is_aborted = true;
@@ -1434,7 +1435,7 @@ quoteFunc(sql_context * context, int argc, sql_value ** argv)
 	case MP_MAP: {
 			char *zText = 0;
 			char const *zBlob = mem_get_blob(argv[0]);
-			int nBlob = sql_value_bytes(argv[0]);
+			int nBlob = mem_get_length(argv[0]);
 			assert(zBlob == mem_get_blob(argv[0]));	/* No encoding change */
 			zText =
 			    (char *)contextMalloc(context,
@@ -1570,7 +1571,7 @@ hexFunc(sql_context * context, int argc, sql_value ** argv)
 	assert(argc == 1);
 	UNUSED_PARAMETER(argc);
 	pBlob = mem_get_blob(argv[0]);
-	n = sql_value_bytes(argv[0]);
+	n = mem_get_length(argv[0]);
 	assert(pBlob == mem_get_blob(argv[0]));	/* No encoding change */
 	z = zHex = contextMalloc(context, ((i64) n) * 2 + 1);
 	if (zHex) {
@@ -1629,7 +1630,7 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
 	zStr = mem_get_ustr(argv[0]);
 	if (zStr == 0)
 		return;
-	nStr = sql_value_bytes(argv[0]);
+	nStr = mem_get_length(argv[0]);
 	assert(zStr == mem_get_ustr(argv[0]));	/* No encoding change */
 	zPattern = mem_get_ustr(argv[1]);
 	if (zPattern == 0) {
@@ -1637,7 +1638,7 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
 		       || sql_context_db_handle(context)->mallocFailed);
 		return;
 	}
-	nPattern = sql_value_bytes(argv[1]);
+	nPattern = mem_get_length(argv[1]);
 	if (nPattern == 0) {
 		assert(! mem_is_null(argv[1]));
 		sql_result_value(context, argv[0]);
@@ -1647,7 +1648,7 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
 	zRep = mem_get_ustr(argv[2]);
 	if (zRep == 0)
 		return;
-	nRep = sql_value_bytes(argv[2]);
+	nRep = mem_get_length(argv[2]);
 	assert(zRep == mem_get_ustr(argv[2]));
 	nOut = nStr + 1;
 	assert(nOut < SQL_MAX_LENGTH);
@@ -1809,7 +1810,7 @@ trim_func_one_arg(struct sql_context *context, sql_value *arg)
 		default_trim = (const unsigned char *) "\0";
 	else
 		default_trim = (const unsigned char *) " ";
-	int input_str_sz = sql_value_bytes(arg);
+	int input_str_sz = mem_get_length(arg);
 	const unsigned char *input_str = mem_get_ustr(arg);
 	uint8_t trim_char_len[1] = { 1 };
 	trim_procedure(context, TRIM_BOTH, default_trim, trim_char_len, 1,
@@ -1835,7 +1836,7 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1,
 	if ((input_str = mem_get_ustr(arg2)) == NULL)
 		return;
 
-	int input_str_sz = sql_value_bytes(arg2);
+	int input_str_sz = mem_get_length(arg2);
 	if (sql_value_type(arg1) == MP_INT || sql_value_type(arg1) == MP_UINT) {
 		uint8_t len_one = 1;
 		bool unused;
@@ -1844,7 +1845,7 @@ trim_func_two_args(struct sql_context *context, sql_value *arg1,
 		trim_procedure(context, n, (const unsigned char *) " ",
 			       &len_one, 1, input_str, input_str_sz);
 	} else if ((trim_set = mem_get_ustr(arg1)) != NULL) {
-		int trim_set_sz = sql_value_bytes(arg1);
+		int trim_set_sz = mem_get_length(arg1);
 		uint8_t *char_len;
 		int char_cnt = trim_prepare_char_len(context, trim_set,
 						     trim_set_sz, &char_len);
@@ -1873,8 +1874,8 @@ trim_func_three_args(struct sql_context *context, sql_value *arg1,
 	    (trim_set = mem_get_ustr(arg2)) == NULL)
 		return;
 
-	int trim_set_sz = sql_value_bytes(arg2);
-	int input_str_sz = sql_value_bytes(arg3);
+	int trim_set_sz = mem_get_length(arg2);
+	int input_str_sz = mem_get_length(arg3);
 	uint8_t *char_len;
 	int char_cnt = trim_prepare_char_len(context, trim_set, trim_set_sz,
 					     &char_len);
@@ -2196,7 +2197,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
 		if (!firstTerm) {
 			if (argc == 2) {
 				zSep = mem_get_str(argv[1]);
-				nSep = sql_value_bytes(argv[1]);
+				nSep = mem_get_length(argv[1]);
 			} else {
 				zSep = ",";
 				nSep = 1;
@@ -2205,7 +2206,7 @@ groupConcatStep(sql_context * context, int argc, sql_value ** argv)
 				sqlStrAccumAppend(pAccum, zSep, nSep);
 		}
 		zVal = mem_get_str(argv[0]);
-		nVal = sql_value_bytes(argv[0]);
+		nVal = mem_get_length(argv[0]);
 		if (zVal)
 			sqlStrAccumAppend(pAccum, zVal, nVal);
 	}
diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
index bed9ae64a..83b0f4f7f 100644
--- a/src/box/sql/mem.c
+++ b/src/box/sql/mem.c
@@ -1341,6 +1341,16 @@ mem_get_binary(const struct Mem *mem, const char **s)
 	return 0;
 }
 
+int
+mem_get_length(const struct Mem *mem)
+{
+	if ((mem->flags & (MEM_Str | MEM_Blob)) == 0)
+		return 0;
+	if ((mem->flags & MEM_Blob) !=0 && (mem->flags & MEM_Zero) != 0)
+		return mem->n + mem->u.nZero;
+	return mem->n;
+}
+
 int
 mem_copy(struct Mem *to, const struct Mem *from)
 {
@@ -2051,41 +2061,6 @@ sql_value_type(sql_value *pVal)
 	return mem_mp_type(mem);
 }
 
-/*
- * The sqlValueBytes() routine returns the number of bytes in the
- * sql_value object assuming that it uses the encoding "enc".
- * The valueBytes() routine is a helper function.
- */
-static SQL_NOINLINE int
-valueBytes(sql_value * pVal)
-{
-	if (mem_convert_to_string(pVal) != 0)
-		return 0;
-	return pVal->n;
-}
-
-int
-sqlValueBytes(sql_value * pVal)
-{
-	Mem *p = (Mem *) pVal;
-	assert((p->flags & MEM_Null) == 0
-	       || (p->flags & (MEM_Str | MEM_Blob)) == 0);
-	if ((p->flags & MEM_Str) != 0) {
-		return p->n;
-	}
-	if ((p->flags & MEM_Blob) != 0) {
-		if (p->flags & MEM_Zero) {
-			return p->n + p->u.nZero;
-		} else {
-			return p->n;
-		}
-	}
-	if (p->flags & MEM_Null)
-		return 0;
-	return valueBytes(pVal);
-}
-
-
 #ifdef SQL_DEBUG
 /*
  * This routine prepares a memory cell for modification by breaking
@@ -2447,53 +2422,6 @@ releaseMemArray(Mem * p, int N)
 	}
 }
 
-int
-sql_value_bytes(sql_value * pVal)
-{
-	return sqlValueBytes(pVal);
-}
-
-/*
- * Return a pointer to static memory containing an SQL NULL value.
- */
-const Mem *
-columnNullValue(void)
-{
-	/* Even though the Mem structure contains an element
-	 * of type i64, on certain architectures (x86) with certain compiler
-	 * switches (-Os), gcc may align this Mem object on a 4-byte boundary
-	 * instead of an 8-byte one. This all works fine, except that when
-	 * running with SQL_DEBUG defined the sql code sometimes assert()s
-	 * that a Mem structure is located on an 8-byte boundary. To prevent
-	 * these assert()s from failing, when building with SQL_DEBUG defined
-	 * using gcc, we force nullMem to be 8-byte aligned using the magical
-	 * __attribute__((aligned(8))) macro.
-	 */
-	static const Mem nullMem
-#if defined(SQL_DEBUG) && defined(__GNUC__)
-	    __attribute__ ((aligned(8)))
-#endif
-	    = {
-		/* .u          = */  {
-		0},
-		    /* .flags      = */ (u16) MEM_Null,
-		    /* .eSubtype   = */ (u8) 0,
-		    /* .field_type = */ field_type_MAX,
-		    /* .n          = */ (int)0,
-		    /* .z          = */ (char *)0,
-		    /* .zMalloc    = */ (char *)0,
-		    /* .szMalloc   = */ (int)0,
-		    /* .uTemp      = */ (u32) 0,
-		    /* .db         = */ (sql *) 0,
-		    /* .xDel       = */ (void (*)(void *))0,
-#ifdef SQL_DEBUG
-		    /* .pScopyFrom = */ (Mem *) 0,
-		    /* .pFiller    = */ (void *)0,
-#endif
-	};
-	return &nullMem;
-}
-
 /*
  * Return true if the Mem object contains a TEXT or BLOB that is
  * too large - whose size exceeds SQL_MAX_LENGTH.
diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
index 8af924d43..b3f994d9d 100644
--- a/src/box/sql/mem.h
+++ b/src/box/sql/mem.h
@@ -363,6 +363,9 @@ mem_get_string0(const struct Mem *mem, const char **s);
 int
 mem_get_binary(const struct Mem *mem, const char **s);
 
+int
+mem_get_length(const struct Mem *mem);
+
 /**
  * Simple type to str convertor. It is used to simplify
  * error reporting.
@@ -381,8 +384,6 @@ mem_mp_type(struct Mem *mem);
 enum mp_type
 sql_value_type(struct Mem *);
 
-int sqlValueBytes(struct Mem *);
-
 #ifdef SQL_DEBUG
 void sqlVdbeMemAboutToChange(struct Vdbe *, struct Mem *);
 int sqlVdbeCheckMemInvariants(struct Mem *);
@@ -436,18 +437,8 @@ releaseMemArray(Mem * p, int N);
 #define MemSetTypeFlag(p, f) \
    ((p)->flags = ((p)->flags&~(MEM_TypeMask|MEM_Zero))|f)
 
-/** Getters. */
-
-int
-sql_value_bytes(struct Mem *);
-
 #define VdbeFrameMem(p) ((Mem *)&((u8 *)p)[ROUND8(sizeof(VdbeFrame))])
 
-const Mem *
-columnNullValue(void);
-
-/** Checkers. */
-
 int sqlVdbeMemTooBig(Mem *);
 
 int sqlMemCompare(const Mem *, const Mem *, const struct coll *);
diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
index 9dfd0e4cc..7b909240f 100644
--- a/src/box/sql/sqlInt.h
+++ b/src/box/sql/sqlInt.h
@@ -433,9 +433,6 @@ sql_stmt_compile(const char *sql, int bytes_count, struct Vdbe *re_prepared,
 int
 sql_step(sql_stmt *);
 
-int
-sql_column_bytes(sql_stmt *, int iCol);
-
 int
 sql_column_bytes16(sql_stmt *, int iCol);
 
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index e76170965..c87b6bfaa 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -443,41 +443,6 @@ sql_data_count(sql_stmt * pStmt)
 	return pVm->nResColumn;
 }
 
-/*
- * Check to see if column iCol of the given statement is valid.  If
- * it is, return a pointer to the Mem for the value of that column.
- * If iCol is not valid, return a pointer to a Mem which has a value
- * of NULL.
- */
-static Mem *
-columnMem(sql_stmt * pStmt, int i)
-{
-	Vdbe *pVm;
-	Mem *pOut;
-
-	pVm = (Vdbe *) pStmt;
-	if (pVm == 0)
-		return (Mem *) columnNullValue();
-	assert(pVm->db);
-	if (pVm->pResultSet != 0 && i < pVm->nResColumn && i >= 0) {
-		pOut = &pVm->pResultSet[i];
-	} else {
-		pOut = (Mem *) columnNullValue();
-	}
-	return pOut;
-}
-
-/**************************** sql_column_  ******************************
- * The following routines are used to access elements of the current row
- * in the result set.
- */
-
-int
-sql_column_bytes(sql_stmt * pStmt, int i)
-{
-	return sql_value_bytes(columnMem(pStmt, i));
-}
-
 char *
 sql_result_to_msgpack(struct sql_stmt *stmt, uint32_t *tuple_size,
 		      struct region *region)
-- 
2.25.1


^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem()
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-03-29 22:57   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 22:57 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Hi! I appreciate the work you did here!

Truly huge patch. But it does the important thing which I think
should give a huge help in the task of SQL code rework.

On 23.03.2021 10:34, Mergen Imeev via Tarantool-patches wrote:
> Currently, vdbe_decode_msgpack_into_mem() creates a MEM that is not
> properly initialized in case msgpack contains MP_EXT, MP_MAP, or
> MP_ARRAY fields. Also, it doesn't set field_type.

AFAIR, field type wasn't set deliberately. Because we use it only for
strictly typed values obtained from formatted space fields. It wasn't
applied to plain non-formatted values on purpose.

Why do you change that?

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 05/53] sql: move MEM-related functions to mem.c/mem.h
  2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 05/53] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
@ 2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 22:58 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

I didn't compare each line of each moved function with its
original code, so I trust you didn't make any functional
changes.

See 2 comments below.

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> new file mode 100644
> index 000000000..62338e1db
> --- /dev/null
> +++ b/src/box/sql/mem.c
> @@ -0,0 +1,2376 @@

<...>

> +
> +
> +#ifdef SQL_DEBUG
> +/*
> + * This routine prepares a memory cell for modification by breaking
> + * its link to a shallow copy and by marking any current shallow
> + * copies of this cell as invalid.
> + *
> + * This is used for testing and debugging only - to make sure shallow
> + * copies are not misused.
> + */
> +void
> +sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)

1. This is a vdbe method, not mem. It takes mem as an argument, but
is a method of vdbe struct. Lets keep it in the old place.

> +{
> +	int i;
> +	Mem *pX;
> +	for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
> +		if (pX->pScopyFrom == pMem) {
> +			pX->flags |= MEM_Undefined;
> +			pX->pScopyFrom = 0;
> +		}
> +	}
> +	pMem->pScopyFrom = 0;
> +}

<...>

> +
> +/*
> + * Print the SQL that was used to generate a VDBE program.
> + */
> +void
> +sqlVdbePrintSql(Vdbe * p)

2. Ditto. Such functions should be moved to vdbe.h/vdbe.c eventually.
And ideally mem.h/mem.c should not know about vdbe anything (probably
can't be done now).

> +{
> +	const char *z = 0;
> +	if (p->zSql) {
> +		z = p->zSql;
> +	} else if (p->nOp >= 1) {
> +		const VdbeOp *pOp = &p->aOp[0];
> +		if (pOp->opcode == OP_Init && pOp->p4.z != 0) {
> +			z = pOp->p4.z;
> +			while (sqlIsspace(*z))
> +				z++;
> +		}
> +	}
> +	if (z)
> +		printf("SQL: [%s]\n", z);
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 06/53] sql: remove unused MEM-related functions
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 06/53] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches
@ 2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 22:58 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch removes remove unused MEM-related functions.

You can simply omit commit message when it is the same as the
title.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 07/53] sql: disable unused code in sql/vdbemem.c
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 07/53] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches
@ 2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 22:58 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

I didn't even know we have vdbemem.c. It should be deleted someday
when we have mem.h/mem.c instead.

> diff --git a/src/box/sql/sqlInt.h b/src/box/sql/sqlInt.h
> index dd8163f5e..b4293d961 100644
> --- a/src/box/sql/sqlInt.h
> +++ b/src/box/sql/sqlInt.h
> @@ -4151,31 +4151,54 @@ sql_expr_new_column(struct sql *db, struct SrcList *src_list, int src_idx,
>  
>  int sqlExprCheckIN(Parse *, Expr *);
>  
> -int sqlStat4ProbeSetValue(Parse *, struct index_def *, UnpackedRecord **, Expr *, int,
> -			      int, int *);
> -int sqlStat4ValueFromExpr(Parse *, Expr *, enum field_type type,
> -			      sql_value **);
> -void sqlStat4ProbeFree(UnpackedRecord *);
> +/* TODO: Enable this function when stat-tables will be revived. */
> +static inline int
> +sqlStat4ProbeSetValue(struct Parse *parse, struct index_def *def,
> +		      struct UnpackedRecord **rec, struct Expr *expr, int n,
> +		      int i, int *out)

You can also make it take '...' instead of all of these arguments individually.
The same for the other functions.

> +{
> +	(void)parse;
> +	(void)def;
> +	(void)rec;
> +	(void)expr;
> +	(void)n;
> +	(void)i;
> +	(void)out;
> +	unreachable();
> +	return 0;
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 08/53] sql: introduce mem_str()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 08/53] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
@ 2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 22:58 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 2 comments below.

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 62338e1db..0ed5e38d4 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -37,6 +37,39 @@
>  #include "box/tuple.h"
>  #include "mpstream/mpstream.h"
>  
> +enum {
> +	BUF_SIZE = 32,
> +};
> +
> +const char *
> +mem_str(const struct Mem *mem)
> +{
> +	char buf[BUF_SIZE];
> +	switch (mem->flags & MEM_PURE_TYPE_MASK) {
> +	case MEM_Null:
> +		return "NULL";
> +	case MEM_Str:
> +		return tt_sprintf("%.*s", mem->n, mem->z);
> +	case MEM_Int:
> +		return tt_sprintf("%lld", mem->u.i);
> +	case MEM_UInt:
> +		return tt_sprintf("%llu", mem->u.u);
> +	case MEM_Real:
> +		sql_snprintf(BUF_SIZE, &buf[0], "%!.15g", mem->u.r);
> +		return tt_sprintf("%s", buf);
> +	case MEM_Blob:
> +		if ((mem->flags & MEM_Subtype) == 0)
> +			return "varbinary";
> +		assert(mem->subtype == SQL_SUBTYPE_MSGPACK);
> +		return mp_str(mem->z);
> +	case MEM_Bool:
> +		return mem->u.b ? "TRUE" : "FALSE";

1. Why are they capital?

> +	default:
> +		break;
> +	}
> +	return "unknown";
> +}
> diff --git a/src/box/sql/vdbetrace.c b/src/box/sql/vdbetrace.c
> index e84bb3192..eceaa953b 100644
> --- a/src/box/sql/vdbetrace.c
> +++ b/src/box/sql/vdbetrace.c
> @@ -147,33 +145,10 @@ sqlVdbeExpandSql(Vdbe * p,	/* The prepared statement being evaluated */
>  			zRawSql += nToken;
>  			nextIndex = idx + 1;
>  			assert(idx > 0 && idx <= p->nVar);
> -			pVar = &p->aVar[idx - 1];
> -			if (pVar->flags & MEM_Null) {
> -				sqlStrAccumAppend(&out, "NULL", 4);
> -			} else if (pVar->flags & MEM_Int) {
> -				sqlXPrintf(&out, "%lld", pVar->u.i);
> -			} else if (pVar->flags & MEM_UInt) {
> -				sqlXPrintf(&out, "%llu", pVar->u.u);
> -			} else if (pVar->flags & MEM_Real) {
> -				sqlXPrintf(&out, "%!.15g", pVar->u.r);
> -			} else if (pVar->flags & MEM_Str) {
> -				int nOut;	/* Number of bytes of the string text to include in output */
> -				nOut = pVar->n;
> -				sqlXPrintf(&out, "'%.*q'", nOut, pVar->z);
> -			} else if (pVar->flags & MEM_Zero) {
> -				sqlXPrintf(&out, "zeroblob(%d)",
> -					       pVar->u.nZero);
> -			} else {
> -				int nOut;	/* Number of bytes of the blob to include in output */
> -				assert(pVar->flags & MEM_Blob);
> -				sqlStrAccumAppend(&out, "x'", 2);
> -				nOut = pVar->n;
> -				for (i = 0; i < nOut; i++) {
> -					sqlXPrintf(&out, "%02x",
> -						       pVar->z[i] & 0xff);
> -				}
> -				sqlStrAccumAppend(&out, "'", 1);
> -			}
> +			const char *value = mem_str(&p->aVar[idx - 1]);
> +			uint32_t size = MIN(strlen(value), sizeof(zBase) - 1);

2. strlen() might be called twice because MIN is a macro.

> +			memcpy(zBase, value, size);
> +			zBase[size] = '\0';
>  		}
>  	}
>  	if (out.accError)

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 11/53] sql: introduce mem_is_*() functions()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 11/53] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:01 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for working on this!

I have a general comment affecting this entire naming schema.
The names seem too long. Besides, we already had some mem_is_*()
and mem_set_*() functions before this patch, which used short
names for the types.

What I mean is 'integer' -> 'int', 'boolean' -> 'bool',
'string' -> 'str', 'binary' -> 'bin', and so on.

For the integers we have several functions because we split
unsigned, signed, and always negative integers. So we would
need more int-like names. For instance,

	mem_set_uint(uint64_t) - for MEM_UInt.
	mem_set_nint(int64_t) - for MEM_Int.
	mem_set_int(int64_t) - for both, checks the sign inside.
	mem_set_sint(int64_t, bool) - for both, takes the sign flag
	                              in the second argument

This can be discussed. The main point - shorter is better IMO. 

See 14 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_is_*() functions that allows to check current
> MEM state.
> 
> Part of #5818
> ---
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index e600a9800..81b537d9b 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -736,8 +736,8 @@ substrFunc(sql_context * context, int argc, sql_value ** argv)
>  		context->is_aborted = true;
>  		return;
>  	}
> -	if (sql_value_is_null(argv[1])
> -	    || (argc == 3 && sql_value_is_null(argv[2]))
> +	if (mem_is_null(argv[1])
> +	    || (argc == 3 && mem_is_null(argv[2]))

1. This is not movement of huge code blocks, it is rather new code
now. And in the new code better use our code style. Such as

- || goes in the end of line, not in the beginning of a next line;
- Unary operators don't have a whitespace after them;
- Comparison with NULL should be explicit, no implicit boot casts.

I mark the places below which I was able to find after a swift look.

>  	    ) {
>  		return;
>  	}
> @@ -1578,13 +1576,13 @@ replaceFunc(sql_context * context, int argc, sql_value ** argv)
>  	assert(zStr == sql_value_text(argv[0]));	/* No encoding change */
>  	zPattern = sql_value_text(argv[1]);
>  	if (zPattern == 0) {
> -		assert(sql_value_is_null(argv[1])
> +		assert(mem_is_null(argv[1])
>  		       || sql_context_db_handle(context)->mallocFailed);
>  		return;
>  	}
>  	nPattern = sql_value_bytes(argv[1]);
>  	if (nPattern == 0) {
> -		assert(! sql_value_is_null(argv[1]));
> +		assert(! mem_is_null(argv[1]));

2. Whitespace after unary operator.

>  		sql_result_value(context, argv[0]);
>  		return;
>  	}
> @@ -2039,7 +2035,7 @@ countStep(sql_context * context, int argc, sql_value ** argv)
>  		return;
>  	}
>  	p = sql_aggregate_context(context, sizeof(*p));
> -	if ((argc == 0 || ! sql_value_is_null(argv[0])) && p) {
> +	if ((argc == 0 || ! mem_is_null(argv[0])) && p) {

3. Ditto.

>  		p->n++;
>  	}
>  }
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index ec6aaab64..abc9291ef 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -37,6 +37,142 @@
>  #include "box/tuple.h"
>  #include "mpstream/mpstream.h"
>  
> +bool
> +mem_is_null(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Null) != 0;
> +}

4. Maybe better move them all to mem.h. These one-liners easily
can be inlined (the ones which are <= 3 lines long could be moved).

> +
> +bool
> +mem_is_unsigned(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_UInt) != 0;
> +}
> +
> +bool
> +mem_is_string(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Str) != 0;
> +}
> +
> +bool
> +mem_is_number(const struct Mem *mem)
> +{
> +	return (mem->flags & (MEM_Real | MEM_Int |MEM_UInt)) != 0;

5. Missed whitespace after the last '|'.

> +}
> +
> +bool
> +mem_is_double(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Real) != 0;
> +}
> +
> +bool
> +mem_is_integer(const struct Mem *mem)
> +{
> +	return (mem->flags & (MEM_Int | MEM_UInt)) != 0;
> +}
> +
> +bool
> +mem_is_boolean(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Bool) != 0;
> +}
> +
> +bool
> +mem_is_binary(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Blob) != 0;
> +}
> +
> +bool
> +mem_is_map(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Blob) != 0 &&
> +	       (mem->flags & MEM_Subtype) != 0 &&
> +	       mem->subtype == SQL_SUBTYPE_MSGPACK &&
> +	       mp_typeof(*mem->z) == MP_MAP;
> +}
> +
> +bool
> +mem_is_array(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Blob) != 0 &&
> +	       (mem->flags & MEM_Subtype) != 0 &&
> +	       mem->subtype == SQL_SUBTYPE_MSGPACK &&
> +	       mp_typeof(*mem->z) == MP_ARRAY;
> +}
> +
> +bool
> +mem_is_aggregate(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Agg) != 0;
> +}
> +
> +bool
> +mem_is_varstring(const struct Mem *mem)
> +{
> +	return (mem->flags & (MEM_Blob | MEM_Str)) != 0;

6. It does not look right to call it varstring if it includes
binary. A string is always binary, but not each binary object
is a string.

Maybe mem_is_bytes()? mem_is_bytearray()?

> +}
> +
> +bool
> +mem_is_frame(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Frame) != 0;
> +}
> +
> +bool
> +mem_is_undefined(const struct Mem *mem)
> +{
> +	return (mem->flags & MEM_Undefined) != 0;
> +}
> +
> +bool
> +mem_is_static(const struct Mem *mem)
> +{
> +	return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> +	       (mem->flags & MEM_Static) != 0;
> +}
> +
> +bool
> +mem_is_ephemeral(const struct Mem *mem)
> +{
> +	return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> +	       (mem->flags & MEM_Ephem) != 0;

7. How can it be that MEM_Ephem is set, but Str/Blob are not?

> +}
> +
> +bool
> +mem_is_dynamic(const struct Mem *mem)
> +{
> +	return (mem->flags & (MEM_Str | MEM_Blob)) != 0 &&
> +	       (mem->flags & MEM_Dyn) != 0;
> +}
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 12712efb4..05e0f78c1 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -1088,10 +1086,10 @@ case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
>  	 * Concatenation operation can be applied only to
>  	 * strings and blobs.
>  	 */
> -	uint32_t str_type_p1 = pIn1->flags & (MEM_Blob | MEM_Str);
> -	uint32_t str_type_p2 = pIn2->flags & (MEM_Blob | MEM_Str);
> -	if (str_type_p1 == 0 || str_type_p2 == 0) {
> -		char *inconsistent_type = str_type_p1 == 0 ?
> +	bool str_type_p1 = mem_is_varstring(pIn1);
> +	bool str_type_p2 = mem_is_varstring(pIn2);

8. They are not types now. Only flags. Should be renamed to something
more appropriate.

> +	if (!str_type_p1 || !str_type_p2) {
> +		char *inconsistent_type = !str_type_p1 ?
>  					  mem_type_to_str(pIn1) :
>  					  mem_type_to_str(pIn2);
>  		diag_set(ClientError, ER_INCONSISTENT_TYPES,
> @@ -1100,7 +1098,7 @@ case OP_Concat: {           /* same as TK_CONCAT, in1, in2, out3 */
>  	}
>  
>  	/* Moreover, both operands must be of the same type. */
> -	if (str_type_p1 != str_type_p2) {
> +	if (mem_is_string(pIn1) != mem_is_string(pIn2)) {

9. I would recommend mem_is_same_type(). Up to you.

>  		diag_set(ClientError, ER_INCONSISTENT_TYPES,
>  			 mem_type_to_str(pIn2), mem_type_to_str(pIn1));
>  		goto abort_due_to_error;
> @@ -1186,14 +1183,16 @@ case OP_Remainder: {           /* same as TK_REM, in1, in2, out3 */
>  	pIn2 = &aMem[pOp->p2];
>  	type2 = numericType(pIn2);
>  	pOut = vdbe_prepare_null_out(p, pOp->p3);
> -	flags = pIn1->flags | pIn2->flags;
> -	if ((flags & MEM_Null)!=0) goto arithmetic_result_is_null;
> +	if (mem_is_null(pIn1) || mem_is_null(pIn2))
> +		goto arithmetic_result_is_null;
>  	if ((type1 & (MEM_Int | MEM_UInt)) != 0 &&
>  	    (type2 & (MEM_Int | MEM_UInt)) != 0) {
>  		iA = pIn1->u.i;
>  		iB = pIn2->u.i;
> -		bool is_lhs_neg = pIn1->flags & MEM_Int;
> -		bool is_rhs_neg = pIn2->flags & MEM_Int;
> +		bool is_lhs_neg = mem_is_integer(pIn1) &&
> +				  !mem_is_unsigned(pIn1);
> +		bool is_rhs_neg = mem_is_integer(pIn2) &&
> +				  !mem_is_unsigned(pIn2);

10. The checks look overcomplicated. Worth adding mem_is_nint()?

>  		bool is_res_neg;
>  		switch( pOp->opcode) {
>  		case OP_Add: {
> @@ -1509,7 +1508,7 @@ case OP_ShiftRight: {           /* same as TK_RSHIFT, in1, in2, out3 */
>  	pIn1 = &aMem[pOp->p1];
>  	pIn2 = &aMem[pOp->p2];
>  	pOut = vdbe_prepare_null_out(p, pOp->p3);
> -	if ((pIn1->flags | pIn2->flags) & MEM_Null) {
> +	if (mem_is_null(pIn1) || mem_is_null(pIn2)) {

11. This is at least third time I see the check of kind "one of them is null".
And I see more below. Probably worth adding a function which would do it more
efficient, in one check: mem_is_any_null(mem1, mem2) or something. Up to you.

> @@ -1757,11 +1756,10 @@ case OP_Ge: {             /* same as TK_GE, jump, in1, in3 */
>  			 * or not both operands are null.
>  			 */
>  			assert(pOp->opcode==OP_Eq || pOp->opcode==OP_Ne);
> -			assert((flags1 & MEM_Cleared)==0);
> +			assert(!mem_is_cleared(pIn1));
>  			assert((pOp->p5 & SQL_JUMPIFNULL)==0);
> -			if ((flags1&flags3&MEM_Null)!=0
> -			    && (flags3&MEM_Cleared)==0
> -				) {
> +			if (mem_is_null(pIn1) && mem_is_null(pIn3) &&

12. You already know pIn1 or pIn3 is NULL from the 'if' above. So it
would be just a bit faster and easier to do mem_is_same_type(). Only
one branch.

> +			    !mem_is_cleared(pIn3)) {
>  				res = 0;  /* Operands are equal */
>  			} else {
>  				res = 1;  /* Operands are not equal */
> @@ -2982,18 +2965,18 @@ case OP_SeekGT: {       /* jump, in3 */
>  		 * the seek, so convert it.
>  		 */
>  		pIn3 = &aMem[int_field];
> -		if ((pIn3->flags & MEM_Null) != 0)
> +		if (mem_is_null(pIn3))
>  			goto skip_truncate;
> -		if ((pIn3->flags & MEM_Str) != 0)
> +		if (mem_is_string(pIn3))
>  			mem_apply_numeric_type(pIn3);
>  		int64_t i;
> -		if ((pIn3->flags & MEM_Int) == MEM_Int) {
> +		if (mem_is_integer(pIn3) && !mem_is_unsigned(pIn3)) {

13. Better be mem_is_nint(), mentioned in one of the previous
comments.

> @@ -3352,11 +3336,11 @@ case OP_FCopy: {     /* out2 */
>  		pIn1 = &aMem[pOp->p1];
>  	}
>  
> -	if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && (pIn1->flags & MEM_Null)) {
> +	if ((pOp->p3 & OPFLAG_NOOP_IF_NULL) && mem_is_null(pIn1)) {

14. Should be explicit != 0.

>  		pOut = vdbe_prepare_null_out(p, pOp->p2);
>  	} else {
>  		assert(memIsValid(pIn1));
> -		assert((pIn1->flags & (MEM_Int | MEM_UInt)) != 0);
> +		assert(mem_is_integer(pIn1));

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 12/53] sql: introduce mem_copy()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 12/53] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:01 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

I appreciate the work you did here!

See 2 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_copy(). This function copies value from source
> MEM to destination MEM. In case value is string or binary and have not
> static allocation type, it is copied to newly allocated memory.
> 
> Part of #5818
> ---
>  src/box/sql/func.c    |  4 ++--
>  src/box/sql/mem.c     | 54 +++++++++++++++++++++++++------------------
>  src/box/sql/mem.h     |  9 +++++++-
>  src/box/sql/vdbeapi.c |  2 +-
>  src/box/sql/vdbeaux.c |  2 +-
>  5 files changed, 44 insertions(+), 27 deletions(-)
> 
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index 81b537d9b..6b6081150 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -2079,13 +2079,13 @@ minmaxStep(sql_context * context, int NotUsed, sql_value ** argv)
>  		bool is_max = (func->flags & SQL_FUNC_MAX) != 0;
>  		cmp = sqlMemCompare(pBest, pArg, pColl);
>  		if ((is_max && cmp < 0) || (!is_max && cmp > 0)) {
> -			sqlVdbeMemCopy(pBest, pArg);

1. Still is "used" in stat4ValueFromExpr().

> +			mem_copy(pBest, pArg);
>  		} else {
>  			sqlSkipAccumulatorLoad(context);
>  		}
>  	} else {
>  		pBest->db = sql_context_db_handle(context);
> -		sqlVdbeMemCopy(pBest, pArg);
> +		mem_copy(pBest, pArg);
>  	}
>  }
>  
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index abc9291ef..f12441d7c 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -257,6 +257,38 @@ mem_destroy(struct Mem *mem)
>  	mem->zMalloc = NULL;
>  }
>  
> +int
> +mem_copy(struct Mem *to, const struct Mem *from)
> +{
> +	mem_clear(to);
> +	to->u = from->u;
> +	to->flags = from->flags;
> +	to->subtype = from->subtype;
> +	to->field_type = from->field_type;
> +	to->n = from->n;
> +	to->z = from->z;
> +	if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
> +		return 0;
> +	if ((to->flags & MEM_Static) != 0)
> +		return 0;
> +	if ((to->flags & (MEM_Zero | MEM_Blob)) == (MEM_Zero | MEM_Blob)) {
> +		if (sqlVdbeMemExpandBlob(to) != 0)
> +			return -1;
> +		return 0;

2. You can make `returnsqlVdbeMemExpandBlob(to);`, no need to check its result.

Also what was wrong with sqlVdbeMemCopy's way of using sqlVdbeMemMakeWriteable?

> +	}
> +	if (to->szMalloc == 0)
> +		to->zMalloc = sqlDbMallocRaw(to->db, to->n);
> +	else
> +		to->zMalloc = sqlDbReallocOrFree(to->db, to->zMalloc, to->n);
> +	if (to->zMalloc == NULL)
> +		return -1;
> +	to->szMalloc = sqlDbMallocSize(to->db, to->zMalloc);
> +	memcpy(to->zMalloc, to->z, to->n);
> +	to->z = to->zMalloc;
> +	to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Subtype);
> +	return 0;
> +}
> +
>  static inline bool
>  mem_has_msgpack_subtype(struct Mem *mem)
>  {

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 13/53] sql: introduce mem_copy_as_ephemeral()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 13/53] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:01 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 3 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch intoduces mem_copy_as_ephemeral(). This function copies value
> from source MEM to destination MEM. In case value is of string or binary
> type and its allocation type is not static, it copied as value with
> ephemeral allocation type.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c  | 58 ++++++++++++++++++++--------------------------
>  src/box/sql/mem.h  | 15 ++++++------
>  src/box/sql/vdbe.c | 10 ++++----
>  3 files changed, 38 insertions(+), 45 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index f12441d7c..30c568970 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -289,6 +289,25 @@ mem_copy(struct Mem *to, const struct Mem *from)
>  	return 0;
>  }
>  
> +int
> +mem_copy_as_ephemeral(struct Mem *to, const struct Mem *from)

1. Why didn't you keep its return type void?

> +{
> +	mem_clear(to);
> +	to->u = from->u;
> +	to->flags = from->flags;
> +	to->subtype = from->subtype;
> +	to->field_type = from->field_type;
> +	to->n = from->n;
> +	to->z = from->z;
> +	if ((to->flags & (MEM_Str | MEM_Blob)) == 0)
> +		return 0;
> +	if ((to->flags & (MEM_Static | MEM_Ephem)) != 0)
> +		return 0;
> +	to->flags &= (MEM_Str | MEM_Blob | MEM_Term | MEM_Zero | MEM_Subtype);
> +	to->flags |= MEM_Ephem;
> +	return 0;
> +}
> @@ -593,9 +598,12 @@ sqlVdbeMemAboutToChange(Vdbe * pVdbe, Mem * pMem)
>  	int i;
>  	Mem *pX;
>  	for (i = 0, pX = pVdbe->aMem; i < pVdbe->nMem; i++, pX++) {
> -		if (pX->pScopyFrom == pMem) {
> -			pX->flags |= MEM_Undefined;
> -			pX->pScopyFrom = 0;
> +		if ((pX->flags & (MEM_Blob | MEM_Str)) != 0 &&
> +		    (pX->flags & (MEM_Ephem | MEM_Static)) == 0) {
> +			if (pX->pScopyFrom == pMem) {
> +				pX->flags |= MEM_Undefined;
> +				pX->pScopyFrom = 0;
> +			}

2. Why did you change that?

>  		}
>  	}
>  	pMem->pScopyFrom = 0;
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 05e0f78c1..55083fb23 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -979,7 +979,7 @@ case OP_Copy: {
>  	pOut = &aMem[pOp->p2];
>  	assert(pOut!=pIn1);
>  	while( 1) {
> -		sqlVdbeMemShallowCopy(pOut, pIn1, MEM_Ephem);
> +		mem_copy_as_ephemeral(pOut, pIn1);
>  		Deephemeralize(pOut);

3. You could turn mem_copy_as_ephemeral + Deephemeralize into mem_copy
the previous commit where mem_copy was introduced.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 15/53] sql: rework vdbe_decode_msgpack_into_mem()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 15/53] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:02   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:02 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 3 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> The original vdbe_decode_msgpack_into_mem() returns a MEM that contains
> string and binary values as ephemeral. This patch renames this function
> to vdbe_decode_msgpack_into_mem_ephemeral() and introduces new
> vdbe_decode_msgpack_into_mem(), which returns a MEM that contains string
> and binary values in newly allocated memory.
> 
> This patch actually changes behavior in this case:

1. Changes how? I don't see any changes in the tests.

> CREATE TABLE t1(m VARBINARY primary key);
> INSERT INTO t1 VALUES(x'6178'), (x'6278'), (x'6379');
> SELECT count(*), substr(m,2,1) AS m FROM t1 GROUP BY m;
> SELECT count(*), substr(m,2,1) AS mx FROM t1 GROUP BY mx;
> 
> But it doesn't change behaviour for this:
> 
> CREATE TABLE t2(m STRING primary key);
> INSERT INTO t2 VALUES('ax'), ('bx'), ('cy');
> SELECT count(*), substr(m,2,1) AS m FROM t2 GROUP BY m;
> SELECT count(*), substr(m,2,1) AS mx FROM t2 GROUP BY mx;
> 
> Part of #5818
> Part of #5890
> ---
>  src/box/sql/mem.c     | 16 +++++++++++++++-
>  src/box/sql/mem.h     | 17 ++++++++++++++++-
>  src/box/sql/vdbe.c    | 18 ------------------
>  src/box/sql/vdbeaux.c |  2 +-
>  4 files changed, 32 insertions(+), 21 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 3d42ac63c..a2316cc90 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -2253,7 +2253,8 @@ sqlVdbeRecordCompareMsgpack(const void *key1,
>  }
>  
>  int
> -vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
> +vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
> +				       uint32_t *len)

2. The function name is getting Java vibes. I propose to rename it to
mem_from_mp_ephemeral() and mem_from_mp() correspondingly. They also should
start taking the mem as a first argument.

>  {
>  	const char *start_buf = buf;
>  	switch (mp_typeof(*buf)) {
> @@ -2354,6 +2355,19 @@ install_blob:
>  	return 0;
>  }
>  
> +int
> +vdbe_decode_msgpack_into_mem(const char *buf, struct Mem *mem, uint32_t *len)
> +{
> +	if (vdbe_decode_msgpack_into_ephemeral_mem(buf, mem, len) != 0)
> +		return -1;
> +	if ((mem->flags & (MEM_Str | MEM_Blob)) != 0) {
> +		assert((mem->flags & MEM_Ephem) != 0);
> +		if (sqlVdbeMemGrow(mem, mem->n, 1) != 0)
> +			return -1;

3. Maybe it is worth adding a function like mem_materialize() or
mem_make_writable() for that kind of work.

> +	}
> +	return 0;
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 18/53] sql: introduce mem_arithmetic()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 18/53] sql: introduce mem_arithmetic() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:02   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:02 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 3 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_arithmetic(). Function mem_arithmetic()
> executes arithmetic operations on the first and the second MEMs and
> writes the result to the third MEM.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c  | 224 +++++++++++++++++++++++++++++++++++++--------
>  src/box/sql/mem.h  |   6 +-
>  src/box/sql/vdbe.c | 111 +---------------------
>  3 files changed, 193 insertions(+), 148 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index f160439c9..6120939d8 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -370,6 +370,192 @@ mem_concat(struct Mem *left, struct Mem *right, struct Mem *result)
>  	return 0;
>  }
>  
> +int
> +mem_arithmetic(const struct Mem *left, const struct Mem *right,
> +	       struct Mem *result, int op)
> +{
> +	sqlVdbeMemSetNull(result);

1. Lets use mem_clear() directly. You delete sqlVdbeMemSetNull() later
anyway.

> +	result->field_type = FIELD_TYPE_NUMBER;
> +	if (((left->flags | right->flags) & MEM_Null) != 0)
> +		return 0;
> +
> +	int64_t il;
> +	bool is_l_neg;
> +	double dl;
> +	uint16_t type_left = 0;

2. Looks surprising when you use 'l' in 3 variables before
and suddenly 'left' here.

Tbh, the old names with A and B looked easier to read. Up to you,
I can live with both.

> +	if ((left->flags & MEM_Real) != 0) {
> +		dl = left->u.r;
> +		type_left = MEM_Real;
> +	} else if ((left->flags & MEM_Int) != 0) {
> +		il = left->u.i;
> +		type_left = MEM_Int;
> +		is_l_neg = true;
> +	} else if ((left->flags & MEM_UInt) != 0) {
> +		il = left->u.i;
> +		type_left = MEM_UInt;
> +		is_l_neg = false;
> +	} else if ((left->flags & (MEM_Str | MEM_Blob)) != 0) {
> +		if (sql_atoi64(left->z, &il, &is_l_neg, left->n) == 0)
> +			type_left = is_l_neg ? MEM_Int : MEM_UInt;
> +		else if (sqlAtoF(left->z, &dl, left->n) != 0)
> +			type_left = MEM_Real;
> +	}
> +
> +	int64_t ir;
> +	bool is_r_neg;
> +	double dr;
> +	uint16_t type_right = 0;
> +	if ((right->flags & MEM_Real) != 0) {
> +		dr = right->u.r;
> +		type_right = MEM_Real;
> +	} else if ((right->flags & MEM_Int) != 0) {
> +		ir = right->u.i;
> +		type_right = MEM_Int;
> +		is_r_neg = true;
> +	} else if ((right->flags & MEM_UInt) != 0) {
> +		ir = right->u.i;
> +		type_right = MEM_UInt;
> +		is_r_neg = false;
> +	} else if ((right->flags & (MEM_Str | MEM_Blob)) != 0) {
> +		if (sql_atoi64(right->z, &ir, &is_r_neg, right->n) == 0)
> +			type_right = is_r_neg ? MEM_Int : MEM_UInt;
> +		else if (sqlAtoF(right->z, &dr, right->n) != 0)
> +			type_right = MEM_Real;
> +	}
> +
> +	if ((type_right & (MEM_Int | MEM_UInt | MEM_Real)) == 0) {
> +		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> +			 mem_str(right), "numeric");
> +		return -1;
> +	}
> +	if ((type_left & (MEM_Int | MEM_UInt | MEM_Real)) == 0) {
> +		diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
> +			 mem_str(left), "numeric");
> +		return -1;
> +	}
> +	if (((type_left | type_right) & MEM_Real) != 0) {
> +		if (type_left == MEM_Int)
> +			dl = (double)il;
> +		else if (type_left == MEM_UInt)
> +			dl = (double)(uint64_t)il;
> +
> +		if (type_right == MEM_Int)
> +			dr = (double)ir;
> +		else if (type_right == MEM_UInt)
> +			dr = (double)(uint64_t)ir;
> +
> +		double dres;
> +		switch(op) {
> +		case OP_Add:
> +			dres = dl + dr;
> +			break;
> +		case OP_Subtract:
> +			dres = dl - dr;
> +			break;
> +		case OP_Multiply:
> +			dres = dl * dr;
> +			break;
> +		case OP_Divide:
> +			if (dr == 0.) {
> +				diag_set(ClientError, ER_SQL_EXECUTE,
> +					 "division by zero");
> +				return -1;
> +			}
> +			dres = dl / dr;
> +			break;
> +		case OP_Remainder: {
> +			int64_t il = (int64_t)dl;
> +			int64_t ir = (int64_t)dr;
> +			if (ir == 0) {
> +				diag_set(ClientError, ER_SQL_EXECUTE,
> +					 "division by zero");
> +				return -1;
> +			}
> +			if (ir == -1)
> +				ir = 1;
> +			dres = (double)(il % ir);
> +			break;
> +		}
> +		default:
> +			unreachable();
> +		}
> +		if (sqlIsNaN(dres))
> +			return 0;
> +		result->u.r = dres;
> +		result->flags = MEM_Real;
> +		return 0;
> +	}
> +	int64_t ires;
> +	/*
> +	 * TODO: This is wrong. Both these flags should already be set. This
> +	 * assignment done to not change behaviour of the function, which
> +	 * is currently bugged.
> +	 */
> +	is_l_neg = (left->flags & MEM_Int) != 0;
> +	is_r_neg = (right->flags & MEM_Int) != 0;
> +	bool is_res_neg;
> +	switch(op) {
> +	case OP_Add:
> +		if (sql_add_int(il, is_l_neg, ir, is_r_neg, &ires,
> +				&is_res_neg) != 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "integer is overflowed");
> +			return -1;
> +		}
> +		break;
> +	case OP_Subtract:
> +		if (sql_sub_int(il, is_l_neg, ir, is_r_neg, &ires,
> +				&is_res_neg) != 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "integer is overflowed");
> +			return -1;
> +		}
> +		break;
> +	case OP_Multiply:
> +		if (sql_mul_int(il, is_l_neg, ir, is_r_neg, &ires,
> +				&is_res_neg) != 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "integer is overflowed");
> +			return -1;
> +		}
> +		break;
> +	case OP_Divide:
> +		if (ir == 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "division by zero");
> +			return -1;
> +		}
> +		if (sql_div_int(il, is_l_neg, ir, is_r_neg, &ires,
> +				&is_res_neg) != 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "integer is overflowed");
> +			return -1;
> +		}
> +		break;
> +	case OP_Remainder: {
> +		if (ir == 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "division by zero");
> +			return -1;
> +		}
> +		if (ir == -1)
> +			ir = 1;
> +		if (sql_rem_int(il, is_l_neg, ir, is_r_neg, &ires,
> +				&is_res_neg) != 0) {
> +			diag_set(ClientError, ER_SQL_EXECUTE,
> +				 "integer is overflowed");
> +			return -1;
> +		}
> +		break;
> +	}
> +	default:
> +		unreachable();
> +	}
> +	result->u.i = ires;
> +	result->flags = is_res_neg ? MEM_Int : MEM_UInt;
> +	return 0;

3. The original code looked shorter and easier to read. Why did you
pad it out so much?

Additionally, since you now have a function for doing arith, you
could make one function for each opcode with common parts as a
static functions in mem.c, and call them right from 'case's in
vdbe.c. So instead of one big case,case,case,case: arith which
also has a switch inside, we would have proper case: plus,
case: multiply, etc. Would be faster and would make each
individual function simpler I think.

The names could be mem_arith_plus(), mem_arith_mul(), mem_arith_minus(),
etc.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 19/53] sql: introduce mem_compare()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 19/53] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:03   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:03 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

I don't understand. Why are there still sqlMemCompare(), sqlBlobCompare()
if you also have mem_compare(), compare_blobs(), compare_numbers()?

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 6120939d8..8119644ed 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -556,6 +556,247 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
>  	return 0;
>  }
>  
> +static int
> +compare_blobs(const struct Mem *left, const struct Mem *right, int *result)> +{
> +	int nl = left->n;
> +	int nr = right->n;
> +	int minlen = MIN(nl, nr);
> +
> +	/*
> +	 * It is possible to have a Blob value that has some non-zero content
> +	 * followed by zero content.  But that only comes up for Blobs formed
> +	 * by the OP_MakeRecord opcode, and such Blobs never get passed into
> +	 * mem_compare().
> +	 */
> +	assert((left->flags & MEM_Zero) == 0 || nl == 0);
> +	assert((right->flags & MEM_Zero) == 0 || nr == 0);
> +
> +	if (left->flags & right->flags & MEM_Zero) {

Please, use explicit != 0, in the other places below too.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 20/53] sql: introduce mem_bitwise()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 20/53] sql: introduce mem_bitwise() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:03   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:03 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_bitwise(). Function mem_bitwise() executes
> bitwise operations with two operands and writes the result to the third
> MEM.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c  | 51 +++++++++++++++++++++++++++++++++++++++++++
>  src/box/sql/mem.h  |  3 +++
>  src/box/sql/vdbe.c | 54 +++-------------------------------------------
>  3 files changed, 57 insertions(+), 51 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 8119644ed..2b455e39f 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -556,6 +556,57 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
>  	return 0;
>  }
>  
> +int
> +mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)

Would be better to split it into separate functions. Also why is OP_BitNot
separated? How is it much different from, say, OP_BitAnd?

Besides, having OP_BitNot in mem.c would allow to make sqlVdbeIntValue static
inside mem.c.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 24/53] sql: introduce mem_set_integer()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 24/53] sql: introduce mem_set_integer() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:04 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 5 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_set_integer(). Function mem_set_integer()
> clears MEM and sets it to given integer value.
> 
> Part of #5818
> ---
>  src/box/sql/func.c    |  6 +--
>  src/box/sql/mem.c     | 88 ++++++++++++++++++-------------------------
>  src/box/sql/mem.h     | 21 ++++-------
>  src/box/sql/vdbe.c    | 10 ++---
>  src/box/sql/vdbeapi.c |  4 +-
>  src/box/sql/vdbeaux.c |  6 +--
>  6 files changed, 56 insertions(+), 79 deletions(-)
> 
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index b61de18d8..8f7550f30 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -278,7 +278,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t *size)
>  			mem_set_double(&val[i], field.dval);
>  			break;
>  		case MP_INT:
> -			mem_set_i64(&val[i], field.ival);
> +			mem_set_integer(&val[i], field.ival, true);

1. It is worth adding a function for setting a negative integer,
like I mentioned in one of the previous emails. Might make such
places easier to read.

>  			break;
>  		case MP_UINT:
>  			mem_set_u64(&val[i], field.ival);
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index da2aa5c94..13a587aba 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -557,8 +566,7 @@ mem_arithmetic(const struct Mem *left, const struct Mem *right,
>  	default:
>  		unreachable();
>  	}
> -	result->u.i = ires;
> -	result->flags = is_res_neg ? MEM_Int : MEM_UInt;
> +	mem_set_integer(result, ires, is_res_neg);

2. mem_set_integer() calls mem_clear(), but you already called
clear for the result in the beginning of this function. Better
keep the old version here. Inside of mem functions you can do
things more efficiently sometimes, without using the public API.

Also there was no field_type set to FIELD_TYPE_INTEGER before. Why
did you change that? It was NUMBER.

> @@ -583,13 +592,13 @@ mem_bitwise(struct Mem *left, struct Mem *right, struct Mem *result, int op)
>  		return -1;
>  	}
>  	if (op == OP_BitAnd) {
> -		result->u.i = l & r;
> -		result->flags = result->u.i < 0 ? MEM_Int : MEM_UInt;
> +		res = l & r;
> +		mem_set_integer(result, res, res < 0);

3. The same. Clear() is called second time and field_type is changed,
but it wasn't before. Why? The same in some other similar places in the
patch.

> @@ -1359,15 +1368,14 @@ vdbe_mem_numerify(struct Mem *mem)
>  	if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real | MEM_Null)) != 0)
>  		return 0;
>  	if ((mem->flags & MEM_Bool) != 0) {
> -		mem->u.u = mem->u.b;
> -		MemSetTypeFlag(mem, MEM_UInt);
> +		mem_set_integer(mem, (int64_t)mem->u.b, false);

4. Why can't you replace it with mem_set_u64()? If this is because you
need FIELD_TYPE_INTEGER, then see the question above why the field
type is set now.

>  		return 0;
>  	}> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index f0b56033a..92845d66d 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -2999,8 +2999,8 @@ case OP_FCopy: {     /* out2 */
>  		assert(mem_is_integer(pIn1));
>  
>  		pOut = vdbe_prepare_null_out(p, pOp->p2);
> -		mem_set_int(pOut, pIn1->u.i, pIn1->flags == MEM_Int);
> -		pOut->field_type = pIn1->field_type;
> +		if (mem_copy(pOut, pIn1) != 0)
> +			goto abort_due_to_error;

5. Why? It couldn't fail before, now it can. It copied just
int before, now it calls the full copy function which looks
like an overkill.

>  	}
>  	break;
>  }

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 25/53] sql: introduce mem_set_unsigned()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 25/53] sql: introduce mem_set_unsigned() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:04 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 13a587aba..f7a1a9b8a 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -1765,7 +1774,7 @@ mem_convert_to_unsigned(struct Mem *mem)
>  	double d = mem->u.r;
>  	if (d < 0.0 || d >= (double)UINT64_MAX)
>  		return -1;
> -	mem_set_u64(mem, (uint64_t) d);
> +	mem_set_unsigned(mem, (uint64_t) d);

Whitespace after unary operator.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 27/53] sql: refactor mem_set_double()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 27/53] sql: refactor mem_set_double() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:04 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 2 comments below.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_set_double(). Function mem_set_double()
> clears MEM and sets it to given unsigned value.

1. It is not about unsigned.

> Part of #5818
> ---
> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index bf3690b7c..49b4e4b1a 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -419,10 +422,6 @@ int sqlVdbeMemClearAndResize(struct Mem * pMem, int n);
>  void
>  mem_set_ptr(struct Mem *mem, void *ptr);
>  
> -/** Set double value and MEM_Real flag. */
> -void
> -mem_set_double(struct Mem *mem, double value);

2. Why do you move these functions and their definitions? What was
wrong with keeping them in place?

> -
>  int
>  sqlVdbeMemSetStr(struct Mem *, const char *, int, u8, void (*)(void *));
>  void
> 

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 28/53] sql: refactor mem_set_*_string()
  2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 28/53] sql: refactor mem_set_*_string() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:05 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 2 comments below.

1. I think it would be better to have mem_set_str_* as a prefix
for these functions. 'string' is too long, and by placing allocation
type in the beginning you complicate grep by mem_set_str_* and
autocompletion when I type 'mem_set_str_' and want to see all the
options.

On 23.03.2021 10:35, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_set_*_string() functions. Functions
> mem_set_*_string() clears MEM and sets it to given string value.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c     | 111 +++++++++++++++++++++++++++++++++---------
>  src/box/sql/mem.h     |  26 +++++++++-
>  src/box/sql/sqlInt.h  |   4 --
>  src/box/sql/vdbe.c    |   5 +-
>  src/box/sql/vdbeapi.c |  47 ++++++++++++++----
>  src/box/sql/vdbeaux.c |  45 +++++++----------
>  6 files changed, 171 insertions(+), 67 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 47a71fb30..91ef7f3c8 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -2780,13 +2849,9 @@ vdbe_decode_msgpack_into_ephemeral_mem(const char *buf, struct Mem *mem,
>  		break;
>  	}
>  	case MP_STR: {
> -		/* XXX u32->int */
> -		mem->n = (int) mp_decode_strl(&buf);
> -		mem->flags = MEM_Str | MEM_Ephem;
> -		mem->field_type = FIELD_TYPE_STRING;
> -install_blob:
> -		mem->z = (char *)buf;
> -		buf += mem->n;
> +		uint32_t len = mp_decode_strl(&buf);
> +		mem_set_ephemeral_string(mem, (char *)buf, len);

2. It adds clear() call which is not necessary here. I would
propose to keep the old code for the sake of speed and
consistency with the other mp types.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 30/53] sql: introduce mem_set_*_binary()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 30/53] sql: introduce mem_set_*_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:05 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 2 comments below.

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 59a378e1b..5ee49cdca 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -410,6 +410,61 @@ mem_copy_string0(struct Mem *mem, const char *value)
>  	return 0;
>  }
>  
> +static inline void
> +mem_set_const_bin(struct Mem *mem, char *value, uint32_t size, int alloc_type)
> +{
> +	assert((alloc_type & (MEM_Static | MEM_Ephem)) != 0);
> +	mem_clear(mem);
> +	mem->z = value;
> +	mem->n = size;
> +	mem->flags = MEM_Blob | alloc_type;
> +	mem->field_type = FIELD_TYPE_VARBINARY;
> +}
> +
> +static inline void
> +mem_set_dyn_bin(struct Mem *mem, char *value, uint32_t size, int alloc_type)
> +{
> +	assert((mem->flags & MEM_Dyn) == 0 || value != mem->z);
> +	assert(mem->szMalloc == 0 || value != mem->zMalloc);
> +	assert(alloc_type == MEM_Dyn || alloc_type == 0);
> +	mem_destroy(mem);

1. Why is it destroy here and clear above?

> +	mem->z = value;
> +	mem->n = size;
> +	mem->flags = MEM_Blob | alloc_type;
> +	mem->field_type = FIELD_TYPE_VARBINARY;
> +	if (alloc_type == MEM_Dyn) {
> +		mem->xDel = sql_free;
> +	} else {
> +		mem->xDel = NULL;
> +		mem->zMalloc = mem->z;
> +		mem->szMalloc = sqlDbMallocSize(mem->db, mem->zMalloc);
> +	}
> +}
> diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
> index 5b5e5b0c8..0e51e4809 100644
> --- a/src/box/sql/vdbeapi.c
> +++ b/src/box/sql/vdbeapi.c
> @@ -183,7 +182,13 @@ sql_result_blob(sql_context * pCtx,
>      )
>  {
>  	assert(n >= 0);
> -	if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
> +	if (xDel == SQL_STATIC)
> +		mem_set_static_binary(pCtx->pOut, (char *)z, n);
> +	else if (xDel == SQL_DYNAMIC)
> +		mem_set_allocated_binary(pCtx->pOut, (char *)z, n);
> +	else if (xDel != SQL_TRANSIENT)
> +		mem_set_dynamic_binary(pCtx->pOut, (char *)z, n);
> +	else if (sqlVdbeMemSetStr(pCtx->pOut, z, n, 0, xDel) != 0)
>  		pCtx->is_aborted = true;

2. It seems to me you need to add a generic mem_set_binary which would
take the xdel argument. Repeating this tree of ifs in each usage place
is not any better. The same for the string API.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 31/53] sql: introduce mem_copy_binary()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 31/53] sql: introduce mem_copy_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:05 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

On 23.03.2021 10:36, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_copy_binary() function. Function
> mem_copy_binary() clears MEM, allocates enough memory and copies given
> binary to allocated memory.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c      | 14 ++++++++++++++
>  src/box/sql/mem.h      |  3 +++
>  src/box/sql/vdbe.c     |  7 ++-----
>  src/box/sql/vdbeapi.c  |  4 ++--
>  src/box/sql/vdbesort.c |  6 +-----
>  5 files changed, 22 insertions(+), 12 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 5ee49cdca..99beec9ad 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -465,6 +465,20 @@ mem_set_allocated_binary(struct Mem *mem, char *value, uint32_t size)
>  	mem_set_dyn_bin(mem, value, size, 0);
>  }
>  
> +int
> +mem_copy_binary(struct Mem *mem, const char *value, uint32_t size)
> +{

What if mem is not a binary now? What if it is a frame? Why don't you clear it?

> +	bool is_own_value = (mem->flags & MEM_Blob) != 0 && mem->z == value;
> +	if (sqlVdbeMemGrow(mem, size, is_own_value) != 0)
> +		return -1;
> +	if (!is_own_value)
> +		memcpy(mem->z, value, size);
> +	mem->n = size;
> +	mem->flags = MEM_Blob;
> +	mem->field_type = FIELD_TYPE_VARBINARY;
> +	return 0;
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  2021-04-09 19:52     ` Mergen Imeev via Tarantool-patches
  0 siblings, 1 reply; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:05 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> index 86da1449c..2e8669138 100644
> --- a/src/box/sql/vdbeaux.c
> +++ b/src/box/sql/vdbeaux.c
> @@ -1305,19 +1305,26 @@ sqlVdbeList(Vdbe * p)
>  			 * has not already been seen.
>  			 */
>  			if (pOp->p4type == P4_SUBPROGRAM) {
> -				int nByte = (nSub + 1) * sizeof(SubProgram *);
>  				int j;
>  				for (j = 0; j < nSub; j++) {
>  					if (apSub[j] == pOp->p4.pProgram)
>  						break;
>  				}
> -				if (j == nSub &&
> -				    sqlVdbeMemGrow(pSub, nByte,
> -						   nSub != 0) == 0) {
> -					apSub = (SubProgram **) pSub->z;
> -					apSub[nSub++] = pOp->p4.pProgram;
> -					pSub->flags |= MEM_Blob;
> -					pSub->n = nSub * sizeof(SubProgram *);
> +				if (j == nSub) {
> +					uint32_t size = sizeof(SubProgram *);
> +					char *value = (char *)&pOp->p4.pProgram;
> +					if (nSub == 0) {
> +						if (mem_copy_binary(pSub, value,
> +								    size) != 0)
> +							return -1;
> +					} else {
> +						assert(0);

What is this assert? And why does not the append work on empty
binaries?

> +						if (mem_append_to_binary(pSub,
> +									 value,
> +									 size) != 0)
> +							return -1;
> +					}
> +					++nSub;
>  				}
>  			}
>  		}
> 

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 34/53] sql: introduce mem_set_*_map() and mem_set_*_array()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 34/53] sql: introduce mem_set_*_map() and mem_set_*_array() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:05 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 7885caaf5..583de00a2 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -508,6 +508,86 @@ mem_append_to_binary(struct Mem *mem, const char *value, uint32_t size)
>  	return 0;
>  }
>  
> +void
> +mem_set_ephemeral_map(struct Mem *mem, char *value, uint32_t size)
> +{
> +	assert(mp_typeof(*value) == MP_MAP);
> +	mem_set_const_bin(mem, value, size, MEM_Ephem);
> +	mem->flags |= MEM_Subtype;
> +	mem->subtype = SQL_SUBTYPE_MSGPACK;
> +	mem->field_type = FIELD_TYPE_MAP;
> +}
> +
> +void
> +mem_set_static_map(struct Mem *mem, char *value, uint32_t size)
> +{
> +	assert(mp_typeof(*value) == MP_MAP);
> +	mem_set_const_bin(mem, value, size, MEM_Static);
> +	mem->flags |= MEM_Subtype;
> +	mem->subtype = SQL_SUBTYPE_MSGPACK;
> +	mem->field_type = FIELD_TYPE_MAP;
> +}
> +
> +void
> +mem_set_dynamic_map(struct Mem *mem, char *value, uint32_t size)

I think I lost the clue of what is the difference between dynamic
and allocated. Maybe worth adding a comment? Or find a better name?

For instance, if one of them is supposed to copy the map, and the
other one to steal its ownership, then you could call them
mem_set_map_copy() and mem_set_map_move(), where move is the same as
C++ move - steal the resource. The same for the others dynamic/allocated
terminology in the function names.

I think I also lost the understanding of static vs ephem by now.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 35/53] sql: introduce mem_set_undefined()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 35/53] sql: introduce mem_set_undefined() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:06 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

On 23.03.2021 10:36, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_set_undefined() function. Function
> mem_set_undefined() invalidates MEM. It does not clears MEM prior to
> invalidating it.

Why does not it clear it?

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 36/53] sql: introduce mem_set_pointer()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 36/53] sql: introduce mem_set_pointer() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:06 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

I think mem_set_ptr() name was better. Was shorter and still easy to
understand.

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index fd2710478..3701741ef 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -595,6 +595,15 @@ mem_set_undefined(struct Mem *mem)
>  	mem->field_type = field_type_MAX;
>  }
>  
> +void
> +mem_set_pointer(struct Mem *mem, void *ptr)
> +{
> +	mem_clear(mem);
> +	mem->flags = MEM_Ptr;
> +	mem->u.p = ptr;
> +	mem->field_type = field_type_MAX;

mem_clear() already sets the type.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 37/53] sql: introduce mem_set_frame()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 37/53] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:06 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 3701741ef..078de0e62 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -604,6 +604,15 @@ mem_set_pointer(struct Mem *mem, void *ptr)
>  	mem->field_type = field_type_MAX;
>  }
>  
> +void
> +mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
> +{
> +	mem_clear(mem);
> +	mem->flags = MEM_Frame;
> +	mem->u.pFrame = frame;
> +	mem->field_type = field_type_MAX;

The type is already installed by mem_clear().

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 38/53] sql: introduce mem_*_aggregate()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 38/53] sql: introduce mem_*_aggregate() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:06 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 2 comments below.

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 078de0e62..0211069c6 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -613,6 +613,29 @@ mem_set_frame(struct Mem *mem, struct VdbeFrame *frame)
>  	mem->field_type = field_type_MAX;
>  }
>  
> +int
> +mem_prepare_aggregate(struct Mem *mem, struct func *func, int size)

1. Why is this called 'prepare' when the other setters are called 'set'?
Maybe use 'set' here as well?

> +{
> +	if (size <= 0) {
> +		mem_clear(mem);
> +		return 0;
> +	}
> +	if (sqlVdbeMemGrow(mem, size, 0) != 0)
> +		return -1;
> +	memset(mem->z, 0, size);
> +	mem->n = size;
> +	mem->flags = MEM_Agg;
> +	mem->u.func = func;
> +	mem->field_type = field_type_MAX;
> +	return 0;
> +}
> +
> +void *
> +mem_get_aggregate(struct Mem *mem)
> +{
> +	return (void *)mem->z;

2. Void cast should work implicitly here I think. But what is
more interesting is why do you even need the getter? The other
mem types don't have getters for the union fields, and it is
fine I suppose. Would be too much. This one does not even check
if the type is an aggregate.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 39/53] sql: introduce mem_set_cleared()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 39/53] sql: introduce mem_set_cleared() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:07 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

I propose to rename that to mem_set_null_clear(). Because
it is a NULL, but with special semantics.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 40/53] sql: move MEM flags to mem.c
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 40/53] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:07 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/mem.h b/src/box/sql/mem.h
> index 70224b55a..8b6f6749d 100644
> --- a/src/box/sql/mem.h
> +++ b/src/box/sql/mem.h
> @@ -563,13 +511,6 @@ columnNullValue(void);
>  
>  int sqlVdbeMemTooBig(Mem *);
>  
> -/* Return TRUE if Mem X contains dynamically allocated content - anything
> - * that needs to be deallocated to avoid a leak.
> - */
> -#define VdbeMemDynamic(X)  \
> -  (((X)->flags&(MEM_Agg|MEM_Dyn|MEM_Frame))!=0)

Why did you remove that? And why don't you have MEM_Agg|MEM_Frame in
mem_is_dynamic()?

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 41/53] sql: introduce mem_convert_to_integer()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 41/53] sql: introduce mem_convert_to_integer() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:07 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 2 comments below.

On 23.03.2021 10:36, imeevma@tarantool.org wrote:
> This patch introduces mem_convert_to_integer() which is used to convert
> a MEM to a MEM that contains integer value.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c  | 114 +++++++++++++++++++++++++++++----------------
>  src/box/sql/mem.h  |   7 ++-
>  src/box/sql/vdbe.c |  19 ++++----
>  3 files changed, 88 insertions(+), 52 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 1209df1ce..b9bcd3d3a 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -695,6 +695,80 @@ mem_set_cleared(struct Mem *mem)

<...>

> +
> +static inline int
> +mem_convert_double_to_integer_lossless(struct Mem *mem)

1. Lets use 'precise'. Here and in other places. And shorten
the names. So it would be mem_convert_bool_to_int(),
mem_convert_to_int(), etc.

> +{
> +	double d = mem->u.r;
> +	if (d < 0 && d >= (double)INT64_MIN && (double)(int64_t)d == d) {
> +		mem_set_integer(mem, (int64_t)d, true);
> +		return 0;
> +	}
> +	if (d >= 0 && d < (double)UINT64_MAX && (double)(uint64_t)d == d) {
> +		mem_set_integer(mem, (int64_t)(uint64_t)d, false);

2. Isn't mem_set_unsigned() faster? Or we can't set FIELD_TYPE_UNSIGNED?

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 44/53] sql: introduce mem_convert_to_string()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 44/53] sql: introduce mem_convert_to_string() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:07 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

See 3 comments below.

On 23.03.2021 10:36, imeevma@tarantool.org wrote:
> This patch introduces mem_convert_to_string(). This function is used to
> convert MEM to MEM contains string value.
> 
> Part of #5818
> ---
>  src/box/sql/mem.c | 197 ++++++++++++++++++++++++++++++----------------
>  src/box/sql/mem.h |   7 +-
>  2 files changed, 133 insertions(+), 71 deletions(-)
> 
> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 8600b5c41..262f48aca 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -818,6 +818,131 @@ mem_convert_to_number(struct Mem *mem)

<...>

> +
> +static inline int
> +mem_convert_boolean_to_string(struct Mem *mem)
> +{
> +	const char *str = mem->u.b ? "TRUE" : "FALSE";
> +	return mem_copy_string0(mem, str);

1. This can be a static string. Because the string
constants are not going anywhere, can ref them as is.

<...>

> +
> +int
> +mem_convert_to_string0(struct Mem *mem)
> +{
> +	if ((mem->flags & MEM_Str) != 0 && (mem->flags & MEM_Term) != 0)

2. Can be done in one check:

	(mem->flags & (MEM_Str | MEM_Term)) == (MEM_Str | MEM_Term)

> +		return 0;
> +	if ((mem->flags & MEM_Str) != 0)
> +		return mem_convert_string_to_string0(mem);
> +	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> +		return mem_convert_integer_to_string(mem);
> +	if ((mem->flags & MEM_Real) != 0)
> +		return mem_convert_double_to_string(mem);
> +	if ((mem->flags & MEM_Bool) != 0)
> +		return mem_convert_boolean_to_string(mem);
> +	if ((mem->flags & MEM_Blob) != 0) {
> +		if ((mem->flags & MEM_Subtype) == 0)
> +			return mem_convert_binary_to_string0(mem);
> +		if (mp_typeof(*mem->z) == MP_MAP)
> +			return mem_convert_map_to_string(mem);
> +		return mem_convert_array_to_string(mem);
> +	}
> +	return -1;
> +}
> +
> +int
> +mem_convert_to_string(struct Mem *mem)

3. Why would you need a not terminated string? The old function
always terminated the strings AFAIS.

> +{
> +	if ((mem->flags & MEM_Str) != 0)
> +		return 0;
> +	if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> +		return mem_convert_integer_to_string(mem);
> +	if ((mem->flags & MEM_Real) != 0)
> +		return mem_convert_double_to_string(mem);
> +	if ((mem->flags & MEM_Bool) != 0)
> +		return mem_convert_boolean_to_string(mem);
> +	if ((mem->flags & MEM_Blob) != 0) {
> +		if ((mem->flags & MEM_Subtype) == 0)
> +			return mem_convert_binary_to_string(mem);
> +		if (mp_typeof(*mem->z) == MP_MAP)
> +			return mem_convert_map_to_string(mem);
> +		return mem_convert_array_to_string(mem);
> +	}
> +	return -1;
> +}
> +
>  int
>  mem_copy(struct Mem *to, const struct Mem *from)
>  {

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 45/53] sql: introduce mem_explicit_cast()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 45/53] sql: introduce mem_explicit_cast() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:08 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

In function names we use subject_action_object pattern. So it
should be mem_cast_explicit(), the same for the implicit cast.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 46/53] sql: introduce mem_implicit_cast()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 46/53] sql: introduce mem_implicit_cast() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:08 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/mem.c b/src/box/sql/mem.c
> index 559bf6121..1baf4c482 100644
> --- a/src/box/sql/mem.c
> +++ b/src/box/sql/mem.c
> @@ -1071,6 +1093,140 @@ mem_explicit_cast(struct Mem *mem, enum field_type type)

<...>

> +
> +int
> +mem_implicit_cast_old(struct Mem *mem, enum field_type type)

What is this?

> +{
> +	if (mem_is_null(mem))
> +		return 0;
> +	switch (type) {
> +	case FIELD_TYPE_UNSIGNED:
> +		if ((mem->flags & MEM_UInt) != 0)
> +			return 0;
> +		if ((mem->flags & MEM_Real) != 0)
> +			return mem_convert_double_to_unsigned_lossless(mem);
> +		if ((mem->flags & MEM_Str) != 0)
> +			return mem_convert_varstring_to_unsigned(mem);
> +		return -1;
> +	case FIELD_TYPE_STRING:
> +		if ((mem->flags & (MEM_Str | MEM_Blob)) != 0)
> +			return 0;
> +		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> +			return mem_convert_integer_to_string(mem);
> +		if ((mem->flags & MEM_Real) != 0)
> +			return mem_convert_double_to_string(mem);
> +		return -1;
> +	case FIELD_TYPE_DOUBLE:
> +		if ((mem->flags & MEM_Real) != 0)
> +			return 0;
> +		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> +			return mem_convert_integer_to_double(mem);
> +		if ((mem->flags & MEM_Str) != 0)
> +			return mem_convert_varstring_to_double(mem);
> +		return -1;
> +	case FIELD_TYPE_INTEGER:
> +		if ((mem->flags & (MEM_Int | MEM_UInt)) != 0)
> +			return 0;
> +		if ((mem->flags & MEM_Str) != 0)
> +			return mem_convert_varstring_to_integer(mem);
> +		if (mem_is_double(mem))
> +			return mem_convert_double_to_integer_lossless(mem);
> +		return -1;
> +	case FIELD_TYPE_BOOLEAN:
> +		if ((mem->flags & MEM_Bool) != 0)
> +			return 0;
> +		return -1;
> +	case FIELD_TYPE_VARBINARY:
> +		if ((mem->flags & MEM_Blob) != 0)
> +			return 0;
> +		return -1;
> +	case FIELD_TYPE_NUMBER:
> +		if ((mem->flags & (MEM_Int | MEM_UInt | MEM_Real)) != 0)
> +			return 0;
> +		if ((mem->flags & MEM_Str) != 0)
> +			return mem_convert_to_number(mem);
> +		return -1;
> +	case FIELD_TYPE_MAP:
> +		if (mem_is_map(mem))
> +			return 0;
> +		return -1;
> +	case FIELD_TYPE_ARRAY:
> +		if (mem_is_array(mem))
> +			return 0;
> +		return -1;
> +	case FIELD_TYPE_SCALAR:
> +		if ((mem->flags & MEM_Blob) != 0 &&
> +		    (mem->flags & MEM_Subtype) != 0)
> +			return -1;
> +		return 0;
> +	default:
> +		break;
> +	}
> +	return -1;
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 47/53] sql: introduce mem_get_integer()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 47/53] sql: introduce mem_get_integer() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:08 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index b644c39d8..0fa0f6ac7 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -1532,10 +1543,11 @@ hexFunc(sql_context * context, int argc, sql_value ** argv)
>  static void
>  zeroblobFunc(sql_context * context, int argc, sql_value ** argv)
>  {
> -	i64 n;
> +	int64_t n;
>  	assert(argc == 1);
>  	UNUSED_PARAMETER(argc);
> -	n = sql_value_int64(argv[0]);
> +	bool unused;
> +	mem_get_integer(argv[0], &n, &unused);

The flag is never used anywhere except one assertion where you can
check the integer value instead. I think you can drop this out
parameter. In future we could add mem_get_int_with_sign() or something
like that if necessary.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 48/53] sql: introduce mem_get_unsigned()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 48/53] sql: introduce mem_get_unsigned() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:08 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

On 23.03.2021 10:36, Mergen Imeev via Tarantool-patches wrote:
> This patch introduces mem_get_unsigned() function which is used to
> receive unsigned value from MEM.
> 
> Part of #5818
> ---
>  src/box/sql/func.c    | 16 +++++++++++-----
>  src/box/sql/mem.c     | 37 +++++++++++++++++++++++++++----------
>  src/box/sql/mem.h     |  6 +++---
>  src/box/sql/sqlInt.h  |  3 ---
>  src/box/sql/vdbeapi.c |  6 ------
>  5 files changed, 41 insertions(+), 27 deletions(-)
> 
> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index 0fa0f6ac7..a851d98f2 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -118,9 +118,12 @@ port_vdbemem_dump_lua(struct port *base, struct lua_State *L, bool is_flat)
>  			luaL_pushint64(L, n);
>  			break;
>  		}
> -		case MP_UINT:
> -			luaL_pushuint64(L, sql_value_uint64(param));
> +		case MP_UINT: {
> +			uint64_t u;
> +			mem_get_unsigned(param, &u);
> +			luaL_pushuint64(L, u);

Maybe we could make 2 functions? One to get the value and ignore
the errors, and the other to get as an out parameter + return an
error?

For instance, mem_to_uint() - returns uint64_t and internally asserts
that the value is correct. And mem_get_uint() works like your version.

The same for the other get functions whose result is often ignored.

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 51/53] sql: introduce mem_get_string0()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 51/53] sql: introduce mem_get_string0() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:08 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index d033dae86..78f4ec3b5 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -54,6 +54,24 @@
>  #include "lua/utils.h"
>  #include "mpstream/mpstream.h"
>  
> +static const char *
> +mem_get_str(struct Mem *mem)
> +{
> +	const char *str;
> +	if (mem_convert_to_string0(mem) != 0 || mem_get_string0(mem, &str) != 0)

1. You have the same code in 2 other files. I think it is worth moving this
to mem.h.

> +		return NULL;
> +	return str;
> +}
> +
> +static const unsigned char *
> +mem_get_ustr(struct Mem *mem)

2. Function called 'get' should not change the value. Maybe 'as'? mem_as_str().

> +{
> +	const char *str;
> +	if (mem_convert_to_string0(mem) != 0 || mem_get_string0(mem, &str) != 0)
> +		return NULL;
> +	return (const unsigned char *)str;
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 52/53] sql: introduce mem_get_binary()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 52/53] sql: introduce mem_get_binary() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:09   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:09 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index 78f4ec3b5..199f3abef 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -72,6 +72,19 @@ mem_get_ustr(struct Mem *mem)
>  	return (const unsigned char *)str;
>  }
>  
> +static const void *
> +mem_get_blob(struct Mem *mem)

The same comments as in the previous patch with strings. Also you
use 'binary' term in the other places, not 'blob'. I would propose to
keep it consistent if we want to switch to 'binary' everywhere.

> +{
> +	const char *s;
> +	if (!mem_is_varstring(mem) && mem_convert_to_string(mem) != 0)
> +		return NULL;
> +	if (ExpandBlob(mem) != 0)
> +		return NULL;
> +	if (mem_get_binary(mem, &s) != 0)
> +		return NULL;
> +	return (const void *)s;
> +}

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 53/53] sql: introduce mem_get_length()
  2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 53/53] sql: introduce mem_get_length() Mergen Imeev via Tarantool-patches
@ 2021-03-29 23:09   ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-03-29 23:09 UTC (permalink / raw)
  To: imeevma, tsafin; +Cc: tarantool-patches

Thanks for the patch!

> diff --git a/src/box/sql/func.c b/src/box/sql/func.c
> index 199f3abef..6687fca3c 100644
> --- a/src/box/sql/func.c
> +++ b/src/box/sql/func.c
> @@ -967,7 +968,7 @@ case_type##ICUFunc(sql_context *context, int argc, sql_value **argv)   \
>  		return;                                                        \
>  	}                                                                      \
>  	z2 = mem_get_str(argv[0]);                                            \
> -	n = sql_value_bytes(argv[0]);                                      \
> +	n = mem_get_length(argv[0]);                                      \

All these \ were aligned before your patchset. Please, keep it straight.

>  	/*                                                                     \
>  	 * Verify that the call to _bytes()                                    \
>  	 * does not invalidate the _text() pointer.                            \

^ permalink raw reply	[flat|nested] 90+ messages in thread

* Re: [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary()
  2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-04-09 19:52     ` Mergen Imeev via Tarantool-patches
  0 siblings, 0 replies; 90+ messages in thread
From: Mergen Imeev via Tarantool-patches @ 2021-04-09 19:52 UTC (permalink / raw)
  To: Vladislav Shpilevoy; +Cc: tarantool-patches

Thank you for the review! My answers below.


On Tue, Mar 30, 2021 at 01:05:44AM +0200, Vladislav Shpilevoy wrote:
> Thanks for the patch!
> 
> > diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
> > index 86da1449c..2e8669138 100644
> > --- a/src/box/sql/vdbeaux.c
> > +++ b/src/box/sql/vdbeaux.c
> > @@ -1305,19 +1305,26 @@ sqlVdbeList(Vdbe * p)
> >  			 * has not already been seen.
> >  			 */
> >  			if (pOp->p4type == P4_SUBPROGRAM) {
> > -				int nByte = (nSub + 1) * sizeof(SubProgram *);
> >  				int j;
> >  				for (j = 0; j < nSub; j++) {
> >  					if (apSub[j] == pOp->p4.pProgram)
> >  						break;
> >  				}
> > -				if (j == nSub &&
> > -				    sqlVdbeMemGrow(pSub, nByte,
> > -						   nSub != 0) == 0) {
> > -					apSub = (SubProgram **) pSub->z;
> > -					apSub[nSub++] = pOp->p4.pProgram;
> > -					pSub->flags |= MEM_Blob;
> > -					pSub->n = nSub * sizeof(SubProgram *);
> > +				if (j == nSub) {
> > +					uint32_t size = sizeof(SubProgram *);
> > +					char *value = (char *)&pOp->p4.pProgram;
> > +					if (nSub == 0) {
> > +						if (mem_copy_binary(pSub, value,
> > +								    size) != 0)
> > +							return -1;
> > +					} else {
> > +						assert(0);
> 
> What is this assert? And why does not the append work on empty
> binaries?
> 
Thank you. This assert did't appear in tests because there is no two or more
triggers executed one after another in tests. I used it for debugging and forgot
to remove it. Fixed. Also, I reworked this part - instead of adding of a new
function (mem_append_to_binary()) I used mem_concat() here. It can be seen in
patch "sql: introduce mem_copy_bin()".

> > +						if (mem_append_to_binary(pSub,
> > +									 value,
> > +									 size) != 0)
> > +							return -1;
> > +					}
> > +					++nSub;
> >  				}
> >  			}
> >  		}
> > 

^ permalink raw reply	[flat|nested] 90+ messages in thread

end of thread, other threads:[~2021-04-09 19:52 UTC | newest]

Thread overview: 90+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-23  9:34 [Tarantool-patches] [PATCH v4 00/53] Move mem-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 01/53] sql: enchance vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
2021-03-29 22:57   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 02/53] sql: disable unused code in sql/analyze.c Mergen Imeev via Tarantool-patches
2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 03/53] sql: disable unused code in sql/legacy.c Mergen Imeev via Tarantool-patches
2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 04/53] sql: remove NULL-termination in OP_ResultRow Mergen Imeev via Tarantool-patches
2021-03-23  9:34 ` [Tarantool-patches] [PATCH v4 05/53] sql: move MEM-related functions to mem.c/mem.h Mergen Imeev via Tarantool-patches
2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 06/53] sql: remove unused MEM-related functions Mergen Imeev via Tarantool-patches
2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 07/53] sql: disable unused code in sql/vdbemem.c Mergen Imeev via Tarantool-patches
2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 08/53] sql: introduce mem_str() Mergen Imeev via Tarantool-patches
2021-03-29 22:58   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 09/53] sql: introduce mem_create() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 10/53] sql: introduce mem_destroy() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 11/53] sql: introduce mem_is_*() functions() Mergen Imeev via Tarantool-patches
2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 12/53] sql: introduce mem_copy() Mergen Imeev via Tarantool-patches
2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 13/53] sql: introduce mem_copy_as_ephemeral() Mergen Imeev via Tarantool-patches
2021-03-29 23:01   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 14/53] sql: rework mem_move() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 15/53] sql: rework vdbe_decode_msgpack_into_mem() Mergen Imeev via Tarantool-patches
2021-03-29 23:02   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 16/53] sql: remove sql_column_to_messagepack() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 17/53] sql: introduce mem_concat() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 18/53] sql: introduce mem_arithmetic() Mergen Imeev via Tarantool-patches
2021-03-29 23:02   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 19/53] sql: introduce mem_compare() Mergen Imeev via Tarantool-patches
2021-03-29 23:03   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 20/53] sql: introduce mem_bitwise() Mergen Imeev via Tarantool-patches
2021-03-29 23:03   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 21/53] sql: introduce mem_bit_not() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 22/53] sql: Initialize MEM in sqlVdbeAllocUnpackedRecord() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 23/53] sql: introduce mem_set_null() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 24/53] sql: introduce mem_set_integer() Mergen Imeev via Tarantool-patches
2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 25/53] sql: introduce mem_set_unsigned() Mergen Imeev via Tarantool-patches
2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 26/53] sql: introduce mem_set_boolean() Mergen Imeev via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 27/53] sql: refactor mem_set_double() Mergen Imeev via Tarantool-patches
2021-03-29 23:04   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 28/53] sql: refactor mem_set_*_string() Mergen Imeev via Tarantool-patches
2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:35 ` [Tarantool-patches] [PATCH v4 29/53] sql: introduce mem_copy_string() Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 30/53] sql: introduce mem_set_*_binary() Mergen Imeev via Tarantool-patches
2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 31/53] sql: introduce mem_copy_binary() Mergen Imeev via Tarantool-patches
2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 32/53] sql: introduce mem_set_zerobinary() Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 33/53] sql: introduce mem_append_to_binary() Mergen Imeev via Tarantool-patches
2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
2021-04-09 19:52     ` Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 34/53] sql: introduce mem_set_*_map() and mem_set_*_array() Mergen Imeev via Tarantool-patches
2021-03-29 23:05   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 35/53] sql: introduce mem_set_undefined() Mergen Imeev via Tarantool-patches
2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 36/53] sql: introduce mem_set_pointer() Mergen Imeev via Tarantool-patches
2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 37/53] sql: introduce mem_set_frame() Mergen Imeev via Tarantool-patches
2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 38/53] sql: introduce mem_*_aggregate() Mergen Imeev via Tarantool-patches
2021-03-29 23:06   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 39/53] sql: introduce mem_set_cleared() Mergen Imeev via Tarantool-patches
2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 40/53] sql: move MEM flags to mem.c Mergen Imeev via Tarantool-patches
2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 41/53] sql: introduce mem_convert_to_integer() Mergen Imeev via Tarantool-patches
2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 42/53] sql: introduce mem_convert_to_double() Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 43/53] sql: introduce mem_convert_to_number() Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 44/53] sql: introduce mem_convert_to_string() Mergen Imeev via Tarantool-patches
2021-03-29 23:07   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 45/53] sql: introduce mem_explicit_cast() Mergen Imeev via Tarantool-patches
2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 46/53] sql: introduce mem_implicit_cast() Mergen Imeev via Tarantool-patches
2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 47/53] sql: introduce mem_get_integer() Mergen Imeev via Tarantool-patches
2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 48/53] sql: introduce mem_get_unsigned() Mergen Imeev via Tarantool-patches
2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 49/53] sql: introduce mem_get_double() Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 50/53] sql: introduce mem_get_boolean() Mergen Imeev via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 51/53] sql: introduce mem_get_string0() Mergen Imeev via Tarantool-patches
2021-03-29 23:08   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 52/53] sql: introduce mem_get_binary() Mergen Imeev via Tarantool-patches
2021-03-29 23:09   ` Vladislav Shpilevoy via Tarantool-patches
2021-03-23  9:36 ` [Tarantool-patches] [PATCH v4 53/53] sql: introduce mem_get_length() Mergen Imeev via Tarantool-patches
2021-03-29 23:09   ` Vladislav Shpilevoy via Tarantool-patches

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox