[tarantool-patches] [PATCH 1/3] sql: fix value of mask to map VDBE memory type

[Nikita Pettik <korablev@tarantool.org>]

Accidentally, mask which is used to map type of VDBE memory cell into
outer API types was replaced with MEM_TypeMask. But value of the latter
is larger then possible values of VDBE memory cells types. Hence, if it
is applied to memory cell, not only pure types is taken into
consideration, but some additional flags (such as MEM_Zero) as well.
Overall, it results in wrong type calculation for zeroed blobs, for
instance. Lets return back previous mask.

Follow-up #3698
Needed for #3544
---
 src/box/sql/vdbeInt.h | 9 +++++++++
 src/box/sql/vdbeapi.c | 2 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index d003d6bb3..933867469 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -253,6 +253,15 @@ struct Mem {
 #define MEM_Zero 0x0000
 #endif
 
+/**
+ * In contrast to Mem_TypeMask, this one allows to get
+ * pure type of memory cell, i.e. without _Dyn/_Zero and other
+ * auxiliary flags.
+ */
+enum {
+	MEM_PURE_TYPE_MASK = 0x1f
+};
+
 
 /* Return TRUE if Mem X contains dynamically allocated content - anything
  * that needs to be deallocated to avoid a leak.
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 7a11bcb0a..250afc20d 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -280,7 +280,7 @@ sql_value_type(sql_value * pVal)
 		SQL_INTEGER,	/* 0x1e */
 		SQL_NULL,	/* 0x1f */
 	};
-	return aType[pVal->flags & MEM_TypeMask];
+	return aType[pVal->flags & MEM_PURE_TYPE_MASK];
 }
 
 /* Make a copy of an sql_value object
-- 
2.15.1