* [tarantool-patches] [PATCH 1/8] sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
2018-12-28 9:34 ` [tarantool-patches] [PATCH 2/8] sql: use field type instead of affinity for type_def Nikita Pettik
` (8 subsequent siblings)
9 siblings, 1 reply; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Code under this define is dead. What is more, it uses affinity, so lets
remove it.
Needed for #3698
---
src/box/sql/parse.y | 2 --
src/box/sql/resolve.c | 20 --------------------
2 files changed, 22 deletions(-)
diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y
index 50bb2ba01..b664a4101 100644
--- a/src/box/sql/parse.y
+++ b/src/box/sql/parse.y
@@ -727,7 +727,6 @@ where_opt(A) ::= WHERE expr(X). {A = X.pExpr;}
////////////////////////// The UPDATE command ////////////////////////////////
//
-%ifndef SQLITE_ENABLE_UPDATE_DELETE_LIMIT
cmd ::= with(C) UPDATE orconf(R) fullname(X) indexed_opt(I) SET setlist(Y)
where_opt(W). {
sqlite3WithPush(pParse, C, 1);
@@ -738,7 +737,6 @@ cmd ::= with(C) UPDATE orconf(R) fullname(X) indexed_opt(I) SET setlist(Y)
pParse->initiateTTrans = true;
sqlite3Update(pParse,X,Y,W,R);
}
-%endif
%type setlist {ExprList*}
%destructor setlist {sql_expr_list_delete(pParse->db, $$);}
diff --git a/src/box/sql/resolve.c b/src/box/sql/resolve.c
index 9a2d6ff4e..c1253ab95 100644
--- a/src/box/sql/resolve.c
+++ b/src/box/sql/resolve.c
@@ -579,26 +579,6 @@ resolveExprStep(Walker * pWalker, Expr * pExpr)
}
#endif
switch (pExpr->op) {
-
-#if defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT)
- /* The special operator TK_ROW means use the rowid for the first
- * column in the FROM clause. This is used by the LIMIT and ORDER BY
- * clause processing on UPDATE and DELETE statements.
- */
- case TK_ROW:{
- SrcList *pSrcList = pNC->pSrcList;
- struct SrcList_item *pItem;
- assert(pSrcList && pSrcList->nSrc == 1);
- pItem = pSrcList->a;
- pExpr->op = TK_COLUMN;
- pExpr->pTab = pItem->pTab;
- pExpr->iTable = pItem->iCursor;
- pExpr->iColumn = -1;
- pExpr->affinity = AFFINITY_INTEGER;
- break;
- }
-#endif /* defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT) */
-
/* A lone identifier is the name of a column.
*/
case TK_ID:{
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 1/8] sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define
2018-12-28 9:34 ` [tarantool-patches] [PATCH 1/8] sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define Nikita Pettik
@ 2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-16 14:25 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2018-12-29 17:42 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Hi! Thanks for the patch!
SQLITE_ENABLE_UPDATE_DELETE_LIMIT still exists
in sql-tap/e_delete.test.lua.
I found out that in this file all tests from line 111
to line 495 are about DELETE LIMIT, so we can remove
them as well, can't we?
On 28/12/2018 12:34, Nikita Pettik wrote:
> Code under this define is dead. What is more, it uses affinity, so lets
> remove it.
>
> Needed for #3698
> ---
> src/box/sql/parse.y | 2 --
> src/box/sql/resolve.c | 20 --------------------
> 2 files changed, 22 deletions(-)
>
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] Re: [PATCH 1/8] sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-01-16 14:25 ` n.pettik
0 siblings, 0 replies; 48+ messages in thread
From: n.pettik @ 2019-01-16 14:25 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 29 Dec 2018, at 19:42, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> Hi! Thanks for the patch!
>
> SQLITE_ENABLE_UPDATE_DELETE_LIMIT still exists
> in sql-tap/e_delete.test.lua.
>
> I found out that in this file all tests from line 111
> to line 495 are about DELETE LIMIT, so we can remove
> them as well, can't we?
Yep, they are anyway disabled so lets remove them as well.
Diff:
diff --git a/test/sql-tap/e_delete.test.lua b/test/sql-tap/e_delete.test.lua
index 84a4e0a22..a58dc87c7 100755
--- a/test/sql-tap/e_delete.test.lua
+++ b/test/sql-tap/e_delete.test.lua
@@ -100,399 +100,5 @@ test:do_delete_tests("e_delete-1.2", {
{9, "DELETE FROM t5 WHERE (SELECT max(x) FROM t6) ;SELECT x FROM t5", {}},
{10, "DELETE FROM t6 WHERE y>'seven' ; SELECT y FROM t6", {"one", "four", "five"}},
})
--- #-------------------------------------------------------------------------
--- # Tests for restrictions on DELETE statements that appear within trigger
--- # programs.
--- #
--- forcedelete test.db2
--- forcedelete test.db3
-
--- MUST_WORK_TEST should be rewritten without spaces or deleted
-if (0 > 0) then
- test:drop_all_tables()
- test:do_execsql_test("e_delete-2.0",
--- ATTACH 'test.db2' AS aux;
--- ATTACH 'test.db3' AS aux2;
- [[
- CREATE TABLE temp.t7(a INT primary key, b INT); INSERT INTO temp.t7 VALUES(1, 2);
- CREATE TABLE main.t7(a INT primary key, b INT); INSERT INTO main.t7 VALUES(3, 4);
- CREATE TABLE aux.t7(a INT primary key, b INT); INSERT INTO aux.t7 VALUES(5, 6);
- CREATE TABLE aux2.t7(a INT primary key, b INT); INSERT INTO aux2.t7 VALUES(7, 8);
- CREATE TABLE main.t8(a INT primary key, b INT); INSERT INTO main.t8 VALUES(1, 2);
- CREATE TABLE aux.t8(a INT primary key, b INT); INSERT INTO aux.t8 VALUES(3, 4);
- CREATE TABLE aux2.t8(a INT primary key, b INT); INSERT INTO aux2.t8 VALUES(5, 6);
- CREATE TABLE aux.t9(a INT primary key, b INT); INSERT INTO aux.t9 VALUES(1, 2);
- CREATE TABLE aux2.t9(a INT primary key, b INT); INSERT INTO aux2.t9 VALUES(3, 4);
- CREATE TABLE aux2.t10(a INT primary key, b INT); INSERT INTO aux2.t10 VALUES(1, 2);]]
- , {})
-
- -- EVIDENCE-OF: R-09681-58560 The table-name specified as part of a
- -- DELETE statement within a trigger body must be unqualified.
- --
- -- EVIDENCE-OF: R-12275-20298 In other words, the schema-name. prefix on
- -- the table name is not allowed within triggers.
- --
-
- test:do_catchsql_test("e_delete-2.1.1",[[
- CREATE TRIGGER tr1 AFTER INSERT ON t1 BEGIN
- DELETE FROM main.t2;
- END;]],
- {1, "qualified table names are not allowed on INSERT, UPDATE, and DELETE statements within triggers"})
- test:do_catchsql_test("e_delete-2.1.2",[[
- CREATE TRIGGER tr1 BEFORE UPDATE ON t2 BEGIN
- DELETE FROM temp.t7 WHERE a=new.a;
- END;]],
- {1, "qualified table names are not allowed on INSERT, UPDATE, and DELETE statements within triggers"})
- test:do_catchsql_test("e_delete-2.1.3",[[
- CREATE TRIGGER tr1 AFTER UPDATE ON t8 BEGIN
- DELETE FROM aux2.t8 WHERE b!=a;
- END;]],
- {1, "qualified table names are not allowed on INSERT, UPDATE, and DELETE statements within triggers"})
- -- EVIDENCE-OF: R-28818-63526 If the table to which the trigger is
- -- attached is not in the temp database, then DELETE statements within
- -- the trigger body must operate on tables within the same database as
- -- it.
- --
- -- This is tested in two parts. First, check that if a table of the
- -- specified name does not exist, an error is raised. Secondly, test
- -- that if tables with the specified name exist in multiple databases,
- -- the local database table is used.
- --
- test:do_delete_tests("e_delete-2.2.1", "-error", " no such table: %s ", {
- {1, [[
- CREATE TRIGGER tr1 AFTER INSERT ON t7 BEGIN
- DELETE FROM t9;
- END;
- INSERT INTO main.t7 VALUES(1, 2);]], {"main.t9"}},
-
- {2, [[
- CREATE TRIGGER aux.tr2 BEFORE UPDATE ON t9 BEGIN
- DELETE FROM t10;
- END;
- UPDATE t9 SET a=1;]], {"aux.t10"}},
- })
- test:do_execsql_test(
- "e_delete-2.2.X",
- [[
- DROP TRIGGER main.tr1;
- DROP TRIGGER aux.tr2;
- ]], {
- -- <e_delete-2.2.X>
-
- -- </e_delete-2.2.X>
- })
-
- test:do_delete_tests("e_delete-2.2.2", {
- {1, [[
- CREATE TRIGGER aux.tr1 AFTER INSERT ON t8 BEGIN
- DELETE FROM t9;
- END;
- INSERT INTO aux.t8 VALUES(1, 2);
-
- SELECT count(*) FROM aux.t9
- UNION ALL
- SELECT count(*) FROM aux2.t9;]], {0, 1}},
- {2, [[
- CREATE TRIGGER main.tr1 AFTER INSERT ON t8 BEGIN
- DELETE FROM t7;
- END;
- INSERT INTO main.t8 VALUES(1, 2);
-
- SELECT count(*) FROM temp.t7
- UNION ALL
- SELECT count(*) FROM main.t7
- UNION ALL
- SELECT count(*) FROM aux.t7
- UNION ALL
- SELECT count(*) FROM aux2.t7;]], {1, 0, 1, 1}}
- })
- -- EVIDENCE-OF: R-31567-38587 If the table to which the trigger is
- -- attached is in the TEMP database, then the unqualified name of the
- -- table being deleted is resolved in the same way as it is for a
- -- top-level statement (by searching first the TEMP database, then the
- -- main database, then any other databases in the order they were
- -- attached).
- --
- test:do_execsql_test(
- "e_delete-2.3.0",
- [[
- DROP TRIGGER aux.tr1;
- DROP TRIGGER main.tr1;
- DELETE FROM main.t8 WHERE oid>1;
- DELETE FROM aux.t8 WHERE oid>1;
- INSERT INTO aux.t9 VALUES(1, 2);
- INSERT INTO main.t7 VALUES(3, 4);
- ]], {
- -- <e_delete-2.3.0>
-
- -- </e_delete-2.3.0>
- })
-
- test:do_execsql_test(
- "e_delete-2.3.1",
- [[
- SELECT count(*) FROM temp.t7 UNION ALL SELECT count(*) FROM main.t7 UNION ALL
- SELECT count(*) FROM aux.t7 UNION ALL SELECT count(*) FROM aux2.t7;
-
- SELECT count(*) FROM main.t8 UNION ALL SELECT count(*) FROM aux.t8
- UNION ALL SELECT count(*) FROM aux2.t8;
-
- SELECT count(*) FROM aux.t9 UNION ALL SELECT count(*) FROM aux2.t9;
-
- SELECT count(*) FROM aux2.t10;
- ]], {
- -- <e_delete-2.3.1>
- 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- -- </e_delete-2.3.1>
- })
-
- test:do_execsql_test(
- "e_delete-2.3.2",
- [[
- CREATE TRIGGER temp.tr1 AFTER INSERT ON t7 BEGIN
- DELETE FROM t7;
- DELETE FROM t8;
- DELETE FROM t9;
- DELETE FROM t10;
- END;
- INSERT INTO temp.t7 VALUES('hello', 'world');
- ]], {
- -- <e_delete-2.3.2>
-
- -- </e_delete-2.3.2>
- })
-
- test:do_execsql_test(
- "e_delete-2.3.3",
- [[
- SELECT count(*) FROM temp.t7 UNION ALL SELECT count(*) FROM main.t7 UNION ALL
- SELECT count(*) FROM aux.t7 UNION ALL SELECT count(*) FROM aux2.t7;
-
- SELECT count(*) FROM main.t8 UNION ALL SELECT count(*) FROM aux.t8
- UNION ALL SELECT count(*) FROM aux2.t8;
-
- SELECT count(*) FROM aux.t9 UNION ALL SELECT count(*) FROM aux2.t9;
-
- SELECT count(*) FROM aux2.t10;
- ]], {
- -- <e_delete-2.3.3>
- 0, 1, 1, 1, 0, 1, 1, 0, 1, 0
- -- </e_delete-2.3.3>
- })
-
- -- EVIDENCE-OF: R-28691-49464 The INDEXED BY and NOT INDEXED clauses are
- -- not allowed on DELETE statements within triggers.
- --
- test:do_execsql_test(
- "e_delete-2.4.0",
- [[
- CREATE INDEX i8 ON t8(a, b);
- ]], {
- -- <e_delete-2.4.0>
-
- -- </e_delete-2.4.0>
- })
-
- test:do_delete_tests("e_delete-2.4", "-error", [[
- the %s %s clause is not allowed on UPDATE or DELETE statements within triggers
- ]], {
- {1, [[
- CREATE TRIGGER tr3 AFTER INSERT ON t8 BEGIN
- DELETE FROM t8 INDEXED BY i8 WHERE a=5;
- END]], {"INDEXED BY"}},
- {2, [[
- CREATE TRIGGER tr3 AFTER INSERT ON t8 BEGIN
- DELETE FROM t8 NOT INDEXED WHERE a=5;
- END;]], {"NOT INDEXED"}},
- })
- -- EVIDENCE-OF: R-64942-06615 The LIMIT and ORDER BY clauses (described
- -- below) are unsupported for DELETE statements within triggers.
- --
- test:do_delete_tests("e_delete-2.5", "-error", [[ near "%s": syntax error ]], {
- {1, [[
- CREATE TRIGGER tr3 AFTER INSERT ON t8 BEGIN
- DELETE FROM t8 LIMIT 10;
- END;]], {"LIMIT"}},
- {2, [[
- CREATE TRIGGER tr3 AFTER INSERT ON t8 BEGIN
- DELETE FROM t8 ORDER BY a LIMIT 5;
- END;]], {"ORDER"}}
- })
- -- EVIDENCE-OF: R-40026-10531 If SQLite is compiled with the
- -- SQLITE_ENABLE_UPDATE_DELETE_LIMIT compile-time option, then the syntax
- -- of the DELETE statement is extended by the addition of optional ORDER
- -- BY and LIMIT clauses:
- --
- -- -- syntax diagram delete-stmt-limited
- --
- test:do_delete_tests("e_delete-3.1", {
- {1, "DELETE FROM t1 LIMIT 5", {}},
- {2, "DELETE FROM t1 LIMIT 5-1 OFFSET 2+2", {}},
- {3, "DELETE FROM t1 LIMIT 2+2, 16/4", {}},
- {4, "DELETE FROM t1 ORDER BY x LIMIT 5", {}},
- {5, "DELETE FROM t1 ORDER BY x LIMIT 5-1 OFFSET 2+2", {}},
- {6, "DELETE FROM t1 ORDER BY x LIMIT 2+2, 16/4", {}},
- {7, "DELETE FROM t1 WHERE x>2 LIMIT 5", {}},
- {8, "DELETE FROM t1 WHERE x>2 LIMIT 5-1 OFFSET 2+2", {}},
- {9, "DELETE FROM t1 WHERE x>2 LIMIT 2+2, 16/4", {}},
- {10, "DELETE FROM t1 WHERE x>2 ORDER BY x LIMIT 5", {}},
- {11, "DELETE FROM t1 WHERE x>2 ORDER BY x LIMIT 5-1 OFFSET 2+2", {}},
- {12, "DELETE FROM t1 WHERE x>2 ORDER BY x LIMIT 2+2, 16/4", {}},
- })
-end
-
-
-
--- MUST_WORK_TEST delete limit syntax
-if (0 > 0) then
- test.drop_all_tables()
- local function rebuild_t1()
- test:catchsql " DROP TABLE t1 "
- test:execsql [[
- CREATE TABLE t1(a INT PRIMARY KEY, b INT );
- INSERT INTO t1 VALUES(1, 'one');
- INSERT INTO t1 VALUES(2, 'two');
- INSERT INTO t1 VALUES(3, 'three');
- INSERT INTO t1 VALUES(4, 'four');
- INSERT INTO t1 VALUES(5, 'five');
- ]]
- end
- -- EVIDENCE-OF: R-44062-08550 If a DELETE statement has a LIMIT clause,
- -- the maximum number of rows that will be deleted is found by evaluating
- -- the accompanying expression and casting it to an integer value.
- --
- rebuild_t1()
- --test:do_delete_tests("e_delete-3.2", "-repair", "rebuild_t1", "-query", "SELECT a FROM t1", {
- test:do_delete_tests("e_delete-3.2", {
- {1, "DELETE FROM t1 LIMIT 3", {4, 5}},
- {2, "DELETE FROM t1 LIMIT 1+1", {3, 4, 5}},
- {3, "DELETE FROM t1 LIMIT '4'", {5}},
- {4, "DELETE FROM t1 LIMIT '1.0'", {2, 3, 4, 5}},
- })
- -- EVIDENCE-OF: R-02661-56399 If the result of the evaluating the LIMIT
- -- clause cannot be losslessly converted to an integer value, it is an
- -- error.
- --
- test:do_delete_tests("e_delete-3.3", "-error", " datatype mismatch ", {
- {1, "DELETE FROM t1 LIMIT 'abc'", {}},
- {2, "DELETE FROM t1 LIMIT NULL", {}},
- {3, "DELETE FROM t1 LIMIT X'ABCD'", {}},
- {4, "DELETE FROM t1 LIMIT 1.2", {}},
-})
- -- EVIDENCE-OF: R-00598-03741 A negative LIMIT value is interpreted as
- -- "no limit".
- --
- test:do_delete_tests("e_delete-3.4", "-repair", "rebuild_t1", "-query", [[
- SELECT a FROM t1
-]], {
- {1, "DELETE FROM t1 LIMIT -1", {}},
- {2, "DELETE FROM t1 LIMIT 2-4", {}},
- {3, "DELETE FROM t1 LIMIT -4.0", {}},
- {4, "DELETE FROM t1 LIMIT 5*-1", {}},
-})
- -- EVIDENCE-OF: R-26377-49195 If the DELETE statement also has an OFFSET
- -- clause, then it is similarly evaluated and cast to an integer value.
- -- Again, it is an error if the value cannot be losslessly converted to
- -- an integer.
- --
- test:do_delete_tests("e_delete-3.5", "-error", " datatype mismatch ", {
- {1, "DELETE FROM t1 LIMIT 1 OFFSET 'abc'", {}},
- {2, "DELETE FROM t1 LIMIT 1 OFFSET NULL", {}},
- {3, "DELETE FROM t1 LIMIT 1 OFFSET X'ABCD'", {}},
- {4, "DELETE FROM t1 LIMIT 1 OFFSET 1.2", {}},
- {5, "DELETE FROM t1 LIMIT 'abc', 1", {}},
- {6, "DELETE FROM t1 LIMIT NULL, 1", {}},
- {7, "DELETE FROM t1 LIMIT X'ABCD', 1", {}},
- {8, "DELETE FROM t1 LIMIT 1.2, 1", {}},
-})
- -- EVIDENCE-OF: R-64004-53814 If there is no OFFSET clause, or the
- -- calculated integer value is negative, the effective OFFSET value is
- -- zero.
- --
- test:do_delete_tests("e_delete-3.6", "-repair", "rebuild_t1", "-query", [[
- SELECT a FROM t1
-]], {
-{"1a", "DELETE FROM t1 LIMIT 3 OFFSET 0", {4, 5}},
-{"1b", "DELETE FROM t1 LIMIT 3", {4, 5}},
-{"1c", "DELETE FROM t1 LIMIT 3 OFFSET -1", {4, 5}},
-{"2a", "DELETE FROM t1 LIMIT 1+1 OFFSET 0", {3, 4, 5}},
-{"2b", "DELETE FROM t1 LIMIT 1+1", {3, 4, 5}},
-{"2c", "DELETE FROM t1 LIMIT 1+1 OFFSET 2-5", {3, 4, 5}},
-{"3a", "DELETE FROM t1 LIMIT '4' OFFSET 0", {5}},
-{"3b", "DELETE FROM t1 LIMIT '4'", {5}},
-{"3c", "DELETE FROM t1 LIMIT '4' OFFSET -1.0", {5}},
-{"4a", "DELETE FROM t1 LIMIT '1.0' OFFSET 0", {2, 3, 4, 5}},
-{"4b", "DELETE FROM t1 LIMIT '1.0'", {2, 3, 4, 5}},
-{"4c", "DELETE FROM t1 LIMIT '1.0' OFFSET -11", {2, 3, 4, 5}},
-})
- -- EVIDENCE-OF: R-48141-52334 If the DELETE statement has an ORDER BY
- -- clause, then all rows that would be deleted in the absence of the
- -- LIMIT clause are sorted according to the ORDER BY. The first M rows,
- -- where M is the value found by evaluating the OFFSET clause expression,
- -- are skipped, and the following N, where N is the value of the LIMIT
- -- expression, are deleted.
- --
- --test:do_delete_tests("e_delete-3.7", "-repair", "rebuild_t1", "-query", "SELECT a FROM t1", {
- test:do_delete_tests("e_delete-3.7",{
- {1, "DELETE FROM t1 ORDER BY b LIMIT 2", {1, 2, 3}},
- {2, "DELETE FROM t1 ORDER BY length(b), a LIMIT 3", {3, 5}},
- {3, "DELETE FROM t1 ORDER BY a DESC LIMIT 1 OFFSET 0", {1, 2, 3, 4}},
- {4, "DELETE FROM t1 ORDER BY a DESC LIMIT 1 OFFSET 1", {1, 2, 3, 5}},
- {5, "DELETE FROM t1 ORDER BY a DESC LIMIT 1 OFFSET 2", {1, 2, 4, 5}},
-})
- -- EVIDENCE-OF: R-64535-08414 If there are less than N rows remaining
- -- after taking the OFFSET clause into account, or if the LIMIT clause
- -- evaluated to a negative value, then all remaining rows are deleted.
- --
- test:do_delete_tests("e_delete-3.8", "-repair", "rebuild_t1", "-query", [[
- SELECT a FROM t1
-]], {
- {1, "DELETE FROM t1 ORDER BY a ASC LIMIT 10", {}},
- {2, "DELETE FROM t1 ORDER BY a ASC LIMIT -1", {}},
- {3, "DELETE FROM t1 ORDER BY a ASC LIMIT 4 OFFSET 2", {1, 2}},
-})
- -- EVIDENCE-OF: R-37284-06965 If the DELETE statement has no ORDER BY
- -- clause, then all rows that would be deleted in the absence of the
- -- LIMIT clause are assembled in an arbitrary order before applying the
- -- LIMIT and OFFSET clauses to determine the subset that are actually
- -- deleted.
- --
- -- In practice, the "arbitrary order" is rowid order.
- --
- test:do_delete_tests("e_delete-3.9", "-repair", "rebuild_t1", "-query", [[
- SELECT a FROM t1
-]], {
- {1, "DELETE FROM t1 LIMIT 2", {3, 4, 5}},
- {2, "DELETE FROM t1 LIMIT 3", {4, 5}},
- {3, "DELETE FROM t1 LIMIT 1 OFFSET 0", {2, 3, 4, 5}},
- {4, "DELETE FROM t1 LIMIT 1 OFFSET 1", {1, 3, 4, 5}},
- {5, "DELETE FROM t1 LIMIT 1 OFFSET 2", {1, 2, 4, 5}},
-})
- -- EVIDENCE-OF: R-07548-13422 The ORDER BY clause on a DELETE statement
- -- is used only to determine which rows fall within the LIMIT. The order
- -- in which rows are deleted is arbitrary and is not influenced by the
- -- ORDER BY clause.
- --
- -- In practice, rows are always deleted in rowid order.
- --
- test:do_delete_tests("e_delete-3.10", "-repair", [[
- rebuild_t1
- catchsql { DROP TABLE t1log }
- execsql {
- CREATE TABLE t1log(x INT );
- CREATE TRIGGER tr1 AFTER DELETE ON t1 BEGIN
- INSERT INTO t1log VALUES(old.a);
- END;
- }
-]], "-query", [[
- SELECT x FROM t1log
-]], {
- {1, "DELETE FROM t1 ORDER BY a DESC LIMIT 2", {4, 5}},
- {2, "DELETE FROM t1 ORDER BY a DESC LIMIT -1", {1, 2, 3, 4, 5}},
- {3, "DELETE FROM t1 ORDER BY a ASC LIMIT 2", {1, 2}},
- {4, "DELETE FROM t1 ORDER BY a ASC LIMIT -1", {1, 2, 3, 4, 5}},
-})
-end
-
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] [PATCH 2/8] sql: use field type instead of affinity for type_def
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
2018-12-28 9:34 ` [tarantool-patches] [PATCH 1/8] sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
2018-12-28 9:34 ` [tarantool-patches] [PATCH 3/8] sql: remove numeric affinity Nikita Pettik
` (7 subsequent siblings)
9 siblings, 1 reply; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Also, this allows to delay affinity assignment to field def until
encoding of table format.
Part of #3698
---
src/box/sql.c | 4 +++-
src/box/sql/build.c | 23 +++++++++++++++++++++--
src/box/sql/parse.y | 26 +++++++++++++-------------
src/box/sql/sqliteInt.h | 5 ++++-
test/sql/types.result | 4 ++--
5 files changed, 43 insertions(+), 19 deletions(-)
diff --git a/src/box/sql.c b/src/box/sql.c
index 8c7607d84..a06c50dca 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -1006,7 +1006,9 @@ sql_encode_table(struct region *region, struct Table *table, uint32_t *size)
action_is_nullable(def->fields[i].nullable_action));
mpstream_encode_str(&stream, field_type_strs[field->type]);
mpstream_encode_str(&stream, "affinity");
- mpstream_encode_uint(&stream, def->fields[i].affinity);
+ enum affinity_type aff =
+ sql_field_type_to_affinity(def->fields[i].type);
+ mpstream_encode_uint(&stream, aff);
mpstream_encode_str(&stream, "is_nullable");
mpstream_encode_bool(&stream, def->fields[i].is_nullable);
mpstream_encode_str(&stream, "nullable_action");
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 49b90b5d0..beaafe1bc 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -501,6 +501,26 @@ sql_affinity_to_field_type(enum affinity_type affinity)
}
}
+enum affinity_type
+sql_field_type_to_affinity(enum field_type field_type)
+{
+ switch (field_type) {
+ case FIELD_TYPE_INTEGER:
+ case FIELD_TYPE_UNSIGNED:
+ return AFFINITY_INTEGER;
+ case FIELD_TYPE_NUMBER:
+ return AFFINITY_REAL;
+ case FIELD_TYPE_STRING:
+ return AFFINITY_TEXT;
+ case FIELD_TYPE_SCALAR:
+ return AFFINITY_BLOB;
+ case FIELD_TYPE_ANY:
+ return AFFINITY_UNDEFINED;
+ default:
+ unreachable();
+ }
+}
+
/*
* Add a new column to the table currently being constructed.
*
@@ -563,8 +583,7 @@ sqlite3AddColumn(Parse * pParse, Token * pName, struct type_def *type_def)
*/
column_def->nullable_action = ON_CONFLICT_ACTION_DEFAULT;
column_def->is_nullable = true;
- column_def->affinity = type_def->type;
- column_def->type = sql_affinity_to_field_type(column_def->affinity);
+ column_def->type = type_def->type;
p->def->field_count++;
pParse->constraintName.n = 0;
}
diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y
index b664a4101..50bc25152 100644
--- a/src/box/sql/parse.y
+++ b/src/box/sql/parse.y
@@ -898,7 +898,7 @@ expr(A) ::= expr(A) COLLATE id(C). {
expr(A) ::= CAST(X) LP expr(E) AS typedef(T) RP(Y). {
spanSet(&A,&X,&Y); /*A-overwrites-X*/
A.pExpr = sqlite3ExprAlloc(pParse->db, TK_CAST, 0, 1);
- A.pExpr->affinity = T.type;
+ A.pExpr->affinity = sql_field_type_to_affinity(T.type);
sqlite3ExprAttachSubtrees(pParse->db, A.pExpr, E.pExpr, 0);
}
%endif SQLITE_OMIT_CAST
@@ -1475,17 +1475,17 @@ wqlist(A) ::= wqlist(A) COMMA nm(X) eidlist_opt(Y) AS LP select(Z) RP. {
}
%endif SQLITE_OMIT_CTE
-/* Primitive types. */
+////////////////////////////// TYPE DECLARATION ///////////////////////////////
%type typedef {struct type_def}
-typedef(A) ::= TEXT . { A.type = AFFINITY_TEXT; }
-typedef(A) ::= BLOB . { A.type = AFFINITY_BLOB; }
-typedef(A) ::= DATE . { A.type = AFFINITY_REAL; }
-typedef(A) ::= TIME . { A.type = AFFINITY_REAL; }
-typedef(A) ::= DATETIME . { A.type = AFFINITY_REAL; }
+typedef(A) ::= TEXT . { A.type = FIELD_TYPE_STRING; }
+typedef(A) ::= BLOB . { A.type = FIELD_TYPE_SCALAR; }
+typedef(A) ::= DATE . { A.type = FIELD_TYPE_NUMBER; }
+typedef(A) ::= TIME . { A.type = FIELD_TYPE_NUMBER; }
+typedef(A) ::= DATETIME . { A.type = FIELD_TYPE_NUMBER; }
%type char_len {int}
typedef(A) ::= CHAR . {
- A.type = AFFINITY_TEXT;
+ A.type = FIELD_TYPE_STRING;
}
char_len(A) ::= LP INTEGER(B) RP . {
@@ -1494,23 +1494,23 @@ char_len(A) ::= LP INTEGER(B) RP . {
}
typedef(A) ::= CHAR char_len(B) . {
- A.type = AFFINITY_TEXT;
+ A.type = FIELD_TYPE_STRING;
(void) B;
}
typedef(A) ::= VARCHAR char_len(B) . {
- A.type = AFFINITY_TEXT;
+ A.type = FIELD_TYPE_STRING;
(void) B;
}
%type number_typedef {struct type_def}
typedef(A) ::= number_typedef(A) .
-number_typedef(A) ::= FLOAT|REAL|DOUBLE . { A.type = AFFINITY_REAL; }
-number_typedef(A) ::= INT|INTEGER . { A.type = AFFINITY_INTEGER; }
+number_typedef(A) ::= FLOAT|REAL|DOUBLE . { A.type = FIELD_TYPE_NUMBER; }
+number_typedef(A) ::= INT|INTEGER . { A.type = FIELD_TYPE_INTEGER; }
%type number_len_typedef {struct type_def}
number_typedef(A) ::= DECIMAL|NUMERIC|NUM number_len_typedef(B) . {
- A.type = AFFINITY_REAL;
+ A.type = FIELD_TYPE_NUMBER;
(void) B;
}
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 4110a5991..b7403d650 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1634,7 +1634,7 @@ struct sqlite3 {
* VARCHAR(<number of chars>).
*/
struct type_def {
- enum affinity_type type;
+ enum field_type type;
};
/*
@@ -3436,6 +3436,9 @@ sql_create_view(struct Parse *parse_context, struct Token *begin,
enum field_type
sql_affinity_to_field_type(enum affinity_type affinity);
+enum affinity_type
+sql_field_type_to_affinity(enum field_type field_type);
+
/**
* Compile view, i.e. create struct Select from
* 'CREATE VIEW...' string, and assign cursors to each table from
diff --git a/test/sql/types.result b/test/sql/types.result
index 1daeb7a8c..915a6341a 100644
--- a/test/sql/types.result
+++ b/test/sql/types.result
@@ -44,8 +44,8 @@ box.sql.execute("CREATE VIEW v1 AS SELECT b + a, b - a FROM t1;")
...
box.space.V1:format()
---
-- [{'affinity': 67, 'type': 'number', 'nullable_action': 'none', 'name': 'b + a',
- 'is_nullable': true}, {'affinity': 67, 'type': 'number', 'nullable_action': 'none',
+- [{'affinity': 69, 'type': 'number', 'nullable_action': 'none', 'name': 'b + a',
+ 'is_nullable': true}, {'affinity': 69, 'type': 'number', 'nullable_action': 'none',
'name': 'b - a', 'is_nullable': true}]
...
-- gh-2494: index's part also features correct declared type.
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 2/8] sql: use field type instead of affinity for type_def
2018-12-28 9:34 ` [tarantool-patches] [PATCH 2/8] sql: use field type instead of affinity for type_def Nikita Pettik
@ 2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-16 14:26 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2018-12-29 17:42 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 1 comment below.
On 28/12/2018 12:34, Nikita Pettik wrote:
> Also, this allows to delay affinity assignment to field def until
> encoding of table format.
>
> Part of #3698
> ---
> src/box/sql.c | 4 +++-
> src/box/sql/build.c | 23 +++++++++++++++++++++--
> src/box/sql/parse.y | 26 +++++++++++++-------------
> src/box/sql/sqliteInt.h | 5 ++++-
> test/sql/types.result | 4 ++--
> 5 files changed, 43 insertions(+), 19 deletions(-)
>
> diff --git a/src/box/sql/build.c b/src/box/sql/build.c
> index 49b90b5d0..beaafe1bc 100644
> --- a/src/box/sql/build.c
> +++ b/src/box/sql/build.c
> @@ -501,6 +501,26 @@ sql_affinity_to_field_type(enum affinity_type affinity)
> }
> }
>
> +enum affinity_type
> +sql_field_type_to_affinity(enum field_type field_type)
If I were you, I would moved this function to field_def.c/.h, but
taking into account that it is a temporary crutch anyway, it is up
to you.
> +{
> + switch (field_type) {
> + case FIELD_TYPE_INTEGER:
> + case FIELD_TYPE_UNSIGNED:
> + return AFFINITY_INTEGER;
> + case FIELD_TYPE_NUMBER:
> + return AFFINITY_REAL;
> + case FIELD_TYPE_STRING:
> + return AFFINITY_TEXT;
> + case FIELD_TYPE_SCALAR:
> + return AFFINITY_BLOB;
> + case FIELD_TYPE_ANY:
> + return AFFINITY_UNDEFINED;
> + default:
> + unreachable();
> + }
> +}
> +
> /*
> * Add a new column to the table currently being constructed.
> *
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 2/8] sql: use field type instead of affinity for type_def
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-01-16 14:26 ` n.pettik
0 siblings, 0 replies; 48+ messages in thread
From: n.pettik @ 2019-01-16 14:26 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
> On 29 Dec 2018, at 19:42, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> Thanks for the patch! See 1 comment below.
>
> If I were you, I would moved this function to field_def.c/.h, but
> taking into account that it is a temporary crutch anyway, it is up
> to you.
Since it is temporary auxiliary function, lets keep it as is.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] [PATCH 3/8] sql: remove numeric affinity
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
2018-12-28 9:34 ` [tarantool-patches] [PATCH 1/8] sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define Nikita Pettik
2018-12-28 9:34 ` [tarantool-patches] [PATCH 2/8] sql: use field type instead of affinity for type_def Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-29 9:01 ` [tarantool-patches] " Konstantin Osipov
` (2 more replies)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 4/8] sql: replace affinity with field type for func Nikita Pettik
` (6 subsequent siblings)
9 siblings, 3 replies; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Numeric affinity in SQLite means the same as real, except that it
forces integer values into floating point representation in case
it can be converted without loss (e.g. 2.0 -> 2).
Since in Tarantool core there is no difference between numeric and real
values (both are stored as values of type NUMBER), lets remove numeric
affinity and use instead real.
The only real pitfall is implicit conversion mentioned above.
What is more, vinyl engine complicates problem since it relies
on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
be able to use iterators from Lua, since they implicitly change type of
1.0 and pass it to the iterator as MP_INT. Solution to this problem is
simple: lets always attempt at encoding floats as ints if conversion
takes place without loss. This is a straightforward approach, but to
implement it we need to care about reversed (decoding) situation.
OP_Column fetches from msgpack field with given number and stores it as
a native VDBE memory object. Type of that memory is based on type of
msgpack value. So, if space field is of type NUMBER and holds value 1,
type of VDBE memory will be INT (after decoding), not float 1.0. As a
result, further calculations may be wrong: for instance, instead of
floating point division, we could get integer division. To cope with
this problem, lets add auxiliary conversion to decoding routine which
uses space format of tuple to be decoded. It is worth mentioning that
ephemeral spaces don't feature space format, so we are going to rely on
type of key parts. Finally, internal VDBE merge sorter also operates on
entries encoded into msgpack. To fix this case, we check type of
ORDER BY/GROUP BY arguments: if they are of type float, we are emitting
additional opcode OP_AffinityReal to force float type after encoding.
Part of #3698
---
src/box/field_def.h | 1 -
src/box/lua/lua_sql.c | 2 +-
src/box/sql.c | 10 +++++++---
src/box/sql/build.c | 1 -
src/box/sql/expr.c | 20 +++++++++++---------
src/box/sql/select.c | 10 +++++++---
src/box/sql/sqliteInt.h | 2 +-
src/box/sql/vdbe.c | 26 ++++++++++++++++++--------
src/box/sql/vdbeaux.c | 19 ++++++++++++++++---
src/box/sql/vdbemem.c | 7 ++-----
test/sql-tap/tkt-80e031a00f.test.lua | 12 ++++++------
11 files changed, 69 insertions(+), 41 deletions(-)
diff --git a/src/box/field_def.h b/src/box/field_def.h
index bd3f9ba45..93e38ea55 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -77,7 +77,6 @@ enum affinity_type {
AFFINITY_UNDEFINED = 0,
AFFINITY_BLOB = 'A',
AFFINITY_TEXT = 'B',
- AFFINITY_NUMERIC = 'C',
AFFINITY_INTEGER = 'D',
AFFINITY_REAL = 'E',
};
diff --git a/src/box/lua/lua_sql.c b/src/box/lua/lua_sql.c
index 5c77b8036..ab0b7f37c 100644
--- a/src/box/lua/lua_sql.c
+++ b/src/box/lua/lua_sql.c
@@ -158,7 +158,7 @@ lbox_sql_create_function(struct lua_State *L)
else if (strcmp(type_arg, "FLOAT") == 0)
type = AFFINITY_REAL;
else if (strcmp(type_arg, "NUM") == 0)
- type = AFFINITY_NUMERIC;
+ type = AFFINITY_REAL;
else if (strcmp(type_arg, "BLOB") == 0)
type = AFFINITY_BLOB;
else
diff --git a/src/box/sql.c b/src/box/sql.c
index a06c50dca..a498cd8fe 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
for (uint32_t i = 0; i < field_count; ++i) {
struct key_part_def *part = &ephemer_key_parts[i];
part->fieldno = i;
- part->type = FIELD_TYPE_SCALAR;
part->nullable_action = ON_CONFLICT_ACTION_NONE;
part->is_nullable = true;
part->sort_order = SORT_ORDER_ASC;
- if (def != NULL && i < def->part_count)
+ if (def != NULL && i < def->part_count) {
+ assert(def->parts[i].type < field_type_MAX);
+ part->type = def->parts[i].type != FIELD_TYPE_ANY ?
+ def->parts[i].type : FIELD_TYPE_SCALAR;
part->coll_id = def->parts[i].coll_id;
- else
+ } else {
part->coll_id = COLL_NONE;
+ part->type = FIELD_TYPE_SCALAR;
+ }
}
struct key_def *ephemer_key_def = key_def_new(ephemer_key_parts,
field_count);
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index beaafe1bc..e51e2db2a 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -492,7 +492,6 @@ sql_affinity_to_field_type(enum affinity_type affinity)
case AFFINITY_INTEGER:
return FIELD_TYPE_INTEGER;
case AFFINITY_REAL:
- case AFFINITY_NUMERIC:
return FIELD_TYPE_NUMBER;
case AFFINITY_TEXT:
return FIELD_TYPE_STRING;
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index b67b22c23..7a0b929a7 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -265,7 +265,7 @@ sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
*/
if (sqlite3IsNumericAffinity(aff1)
|| sqlite3IsNumericAffinity(aff2)) {
- return AFFINITY_NUMERIC;
+ return AFFINITY_REAL;
} else {
return AFFINITY_BLOB;
}
@@ -2168,12 +2168,10 @@ sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
op = p->op2;
switch (op) {
case TK_INTEGER:{
- return aff == AFFINITY_INTEGER
- || aff == AFFINITY_NUMERIC;
+ return aff == AFFINITY_INTEGER;
}
case TK_FLOAT:{
- return aff == AFFINITY_REAL
- || aff == AFFINITY_NUMERIC;
+ return aff == AFFINITY_REAL;
}
case TK_STRING:{
return aff == AFFINITY_TEXT;
@@ -2185,7 +2183,7 @@ sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
assert(p->iTable >= 0); /* p cannot be part of a CHECK constraint */
return p->iColumn < 0
&& (aff == AFFINITY_INTEGER
- || aff == AFFINITY_NUMERIC);
+ || aff == AFFINITY_REAL);
}
default:{
return 0;
@@ -3700,6 +3698,10 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
sqlite3VdbeAddOp3(v, OP_Column,
pAggInfo->sortingIdxPTab,
pCol->iSorterColumn, target);
+ if (pCol->space_def->fields[pExpr->iAgg].type ==
+ FIELD_TYPE_NUMBER)
+ sqlite3VdbeAddOp1(v, OP_RealAffinity,
+ target);
return target;
}
/* Otherwise, fall thru into the TK_COLUMN case */
@@ -3885,14 +3887,14 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
testcase(regFree1 == 0);
testcase(regFree2 == 0);
if (op != TK_CONCAT)
- pExpr->affinity = AFFINITY_NUMERIC;
+ pExpr->affinity = AFFINITY_REAL;
else
pExpr->affinity = AFFINITY_TEXT;
break;
}
case TK_UMINUS:{
Expr *pLeft = pExpr->pLeft;
- pExpr->affinity = AFFINITY_NUMERIC;
+ pExpr->affinity = AFFINITY_REAL;
assert(pLeft);
if (pLeft->op == TK_INTEGER) {
expr_code_int(pParse, pLeft, true, target);
@@ -4184,7 +4186,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
target);
}
case TK_UPLUS:{
- pExpr->affinity = AFFINITY_NUMERIC;
+ pExpr->affinity = AFFINITY_REAL;
return sqlite3ExprCodeTarget(pParse, pExpr->pLeft,
target);
}
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 02ee225f1..40336e679 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1427,6 +1427,8 @@ sql_expr_list_to_key_info(struct Parse *parse, struct ExprList *list, int start)
sql_expr_coll(parse, item->pExpr, &unused, &id);
part->coll_id = id;
part->sort_order = item->sort_order;
+ enum affinity_type aff = sqlite3ExprAffinity(item->pExpr);
+ part->type =sql_affinity_to_field_type(aff);
}
return key_info;
}
@@ -1723,7 +1725,6 @@ generateColumnNames(Parse * pParse, /* Parser context */
SQLITE_TRANSIENT);
break;
case AFFINITY_REAL:
- case AFFINITY_NUMERIC:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "NUMERIC",
SQLITE_TRANSIENT);
break;
@@ -5789,8 +5790,11 @@ sqlite3Select(Parse * pParse, /* The parser context */
/* If the output is destined for a temporary table, open that table.
*/
if (pDest->eDest == SRT_EphemTab) {
- sqlite3VdbeAddOp2(v, OP_OpenTEphemeral, pDest->reg_eph,
- pEList->nExpr + 1);
+ struct sql_key_info *key_info =
+ sql_expr_list_to_key_info(pParse, pEList, 0);
+ sqlite3VdbeAddOp4(v, OP_OpenTEphemeral, pDest->reg_eph,
+ pEList->nExpr + 1, 0, (char *)key_info,
+ P4_KEYINFO);
sqlite3VdbeAddOp3(v, OP_IteratorOpen, pDest->iSDParm, 0,
pDest->reg_eph);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index b7403d650..e2d630929 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1788,7 +1788,7 @@ struct Savepoint {
#define SAVEPOINT_RELEASE 1
#define SAVEPOINT_ROLLBACK 2
-#define sqlite3IsNumericAffinity(X) ((X)>=AFFINITY_NUMERIC)
+#define sqlite3IsNumericAffinity(X) ((X)>=AFFINITY_INTEGER)
/*
* The AFFINITY_MASK values masks off the significant bits of an
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index e6b413c70..4345af24e 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -310,7 +310,6 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
*
* AFFINITY_INTEGER:
* AFFINITY_REAL:
- * AFFINITY_NUMERIC:
* Try to convert mem to an integer representation or a
* floating-point representation if an integer representation
* is not possible. Note that the integer representation is
@@ -345,13 +344,9 @@ mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
}
return sqlite3VdbeMemIntegerify(record, false);
case AFFINITY_REAL:
- if ((record->flags & MEM_Real) == MEM_Real)
- return 0;
- return sqlite3VdbeMemRealify(record);
- case AFFINITY_NUMERIC:
if ((record->flags & (MEM_Real | MEM_Int)) != 0)
return 0;
- return sqlite3VdbeMemNumerify(record);
+ return sqlite3VdbeMemRealify(record);
case AFFINITY_TEXT:
/*
* Only attempt the conversion to TEXT if there is
@@ -2005,7 +2000,6 @@ case OP_Cast: { /* in1 */
assert(pOp->p2>=AFFINITY_BLOB && pOp->p2<=AFFINITY_REAL);
testcase( pOp->p2==AFFINITY_TEXT);
testcase( pOp->p2==AFFINITY_BLOB);
- testcase( pOp->p2==AFFINITY_NUMERIC);
testcase( pOp->p2==AFFINITY_INTEGER);
testcase( pOp->p2==AFFINITY_REAL);
pIn1 = &aMem[pOp->p1];
@@ -2173,7 +2167,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
} else {
/* Neither operand is NULL. Do a comparison. */
affinity = pOp->p5 & AFFINITY_MASK;
- if (affinity>=AFFINITY_NUMERIC) {
+ if (affinity>=AFFINITY_INTEGER) {
if ((flags1 | flags3)&MEM_Str) {
if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
applyNumericAffinity(pIn1,0);
@@ -2768,6 +2762,22 @@ case OP_Column: {
pDest->flags = MEM_Blob|MEM_Ephem|MEM_Subtype;
pDest->subtype = SQL_SUBTYPE_MSGPACK;
}
+ if ((pDest->flags & MEM_Int) != 0 &&
+ pC->eCurType == CURTYPE_TARANTOOL) {
+ enum field_type f = FIELD_TYPE_ANY;
+ /*
+ * Ephemeral spaces feature only one index
+ * covering all fields, but ephemeral spaces
+ * lack format. So, we can fetch type from
+ * key parts.
+ */
+ if (pC->uc.pCursor->curFlags & BTCF_TEphemCursor)
+ f = pC->uc.pCursor->index->def->key_def->parts[p2].type;
+ else if (pC->uc.pCursor->curFlags & BTCF_TaCursor)
+ f = pC->uc.pCursor->space->def->fields[p2].type;
+ if (f == FIELD_TYPE_NUMBER)
+ sqlite3VdbeMemSetDouble(pDest, pDest->u.i);
+ }
/*
* Add 0 termination (at most for strings)
* Not sure why do we check MEM_Ephem
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index fc805e3aa..08eaddbfc 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -3561,17 +3561,30 @@ sqlite3VdbeMsgpackRecordPut(u8 * pBuf, Mem * pRec, u32 n)
assert(n != 0);
do {
assert(memIsValid(pRec));
+ int64_t i;
if (pRec->flags & MEM_Null) {
zNewRecord = mp_encode_nil(zNewRecord);
} else if (pRec->flags & MEM_Real) {
+ /*
+ * We can't pass to INT iterator float
+ * value. Hence, if floating point value
+ * lacks fractional component, we can
+ * encode it as INT and successfully
+ * pass to INT iterator.
+ */
+ i = pRec->u.r;
+ if (i == pRec->u.r)
+ goto encode_int;
zNewRecord = mp_encode_double(zNewRecord, pRec->u.r);
} else if (pRec->flags & MEM_Int) {
- if (pRec->u.i >= 0) {
+ i = pRec->u.i;
+encode_int:
+ if (i >= 0) {
zNewRecord =
- mp_encode_uint(zNewRecord, pRec->u.i);
+ mp_encode_uint(zNewRecord, i);
} else {
zNewRecord =
- mp_encode_int(zNewRecord, pRec->u.i);
+ mp_encode_int(zNewRecord, i);
}
} else if (pRec->flags & MEM_Str) {
zNewRecord =
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index 22beba8be..bb4d91aed 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -602,8 +602,7 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
{
if (pMem->flags & MEM_Null)
return SQLITE_OK;
- if ((pMem->flags & MEM_Blob) != 0 &&
- (aff == AFFINITY_REAL || aff == AFFINITY_NUMERIC)) {
+ if ((pMem->flags & MEM_Blob) != 0 && aff == AFFINITY_REAL) {
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Real);
pMem->u.r = pMem->u.i;
@@ -626,8 +625,6 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
return 0;
}
return SQLITE_ERROR;
- case AFFINITY_NUMERIC:
- return sqlite3VdbeMemNumerify(pMem);
case AFFINITY_INTEGER:
if ((pMem->flags & MEM_Blob) != 0) {
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i,
@@ -1319,7 +1316,7 @@ valueFromExpr(sqlite3 * db, /* The database connection */
}
if ((op == TK_INTEGER || op == TK_FLOAT)
&& affinity == AFFINITY_BLOB) {
- sqlite3ValueApplyAffinity(pVal, AFFINITY_NUMERIC);
+ sqlite3ValueApplyAffinity(pVal, AFFINITY_REAL);
} else {
sqlite3ValueApplyAffinity(pVal, affinity);
}
diff --git a/test/sql-tap/tkt-80e031a00f.test.lua b/test/sql-tap/tkt-80e031a00f.test.lua
index 0cbdb47a2..2d4f81798 100755
--- a/test/sql-tap/tkt-80e031a00f.test.lua
+++ b/test/sql-tap/tkt-80e031a00f.test.lua
@@ -346,7 +346,7 @@ test:do_catchsql_test(
SELECT 'hello' IN t1
]], {
-- <tkt-80e031a00f.27>
- 1, 'Type mismatch: can not convert hello to numeric'
+ 1, 'Type mismatch: can not convert hello to real'
-- </tkt-80e031a00f.27>
})
@@ -356,7 +356,7 @@ test:do_catchsql_test(
SELECT 'hello' NOT IN t1
]], {
-- <tkt-80e031a00f.28>
- 1, 'Type mismatch: can not convert hello to numeric'
+ 1, 'Type mismatch: can not convert hello to real'
-- </tkt-80e031a00f.28>
})
@@ -380,23 +380,23 @@ test:do_execsql_test(
-- </tkt-80e031a00f.30>
})
-test:do_execsql_test(
+test:do_catchsql_test(
"tkt-80e031a00f.31",
[[
SELECT x'303132' IN t1
]], {
-- <tkt-80e031a00f.31>
- 0
+ 1, 'Type mismatch: can not convert 012 to real'
-- </tkt-80e031a00f.31>
})
-test:do_execsql_test(
+test:do_catchsql_test(
"tkt-80e031a00f.32",
[[
SELECT x'303132' NOT IN t1
]], {
-- <tkt-80e031a00f.32>
- 1
+ 1, 'Type mismatch: can not convert 012 to real'
-- </tkt-80e031a00f.32>
})
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2018-12-28 9:34 ` [tarantool-patches] [PATCH 3/8] sql: remove numeric affinity Nikita Pettik
@ 2018-12-29 9:01 ` Konstantin Osipov
2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-09 8:20 ` Konstantin Osipov
2 siblings, 0 replies; 48+ messages in thread
From: Konstantin Osipov @ 2018-12-29 9:01 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
* Nikita Pettik <korablev@tarantool.org> [18/12/28 12:36]:
> The only real pitfall is implicit conversion mentioned above.
> What is more, vinyl engine complicates problem since it relies
> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
> be able to use iterators from Lua, since they implicitly change type of
> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
> simple: lets always attempt at encoding floats as ints if conversion
> takes place without loss. This is a straightforward approach, but to
> implement it we need to care about reversed (decoding) situation.
I think the issue is more complicated.
First of all, if SQL column type is INT, Tarantool column type
should be INT, and vinyl/memtx index type should be INT. This
is the first thing to do/ensure and this will remove INT data type
from consideration altogether.
Now, for REAL data type, Tarantool data type should be NUMBER, since in
Tarantool we have no separate data type for REAL, but
one should *always* use MP_FLOAT/MP_DOUBLE encoding, even for
integer values within this type - to simplify and speed up
comparison functions.
Finally, to Vinyl. It seems it has a bug in a way that it behaves
differently from memtx, and differently depending on whether the
value is in memory or on disk.
In other words, imagine we don't have any data on disk and bloom
filters are not used. Then the value '1' searched by key '1.0'
will be found. Otherwise not. I believe this should be reported as
a separate bug in vinyl.
--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2018-12-28 9:34 ` [tarantool-patches] [PATCH 3/8] sql: remove numeric affinity Nikita Pettik
2018-12-29 9:01 ` [tarantool-patches] " Konstantin Osipov
@ 2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-09 8:26 ` Konstantin Osipov
2019-01-16 14:26 ` n.pettik
2019-01-09 8:20 ` Konstantin Osipov
2 siblings, 2 replies; 48+ messages in thread
From: Vladislav Shpilevoy @ 2018-12-29 17:42 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 5 comments below.
On 28/12/2018 12:34, Nikita Pettik wrote:
> Numeric affinity in SQLite means the same as real, except that it
> forces integer values into floating point representation in case
> it can be converted without loss (e.g. 2.0 -> 2).
> Since in Tarantool core there is no difference between numeric and real
> values (both are stored as values of type NUMBER), lets remove numeric
> affinity and use instead real.
>
> The only real pitfall is implicit conversion mentioned above.
> What is more, vinyl engine complicates problem since it relies
> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
> be able to use iterators from Lua, since they implicitly change type of
> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
> simple: lets always attempt at encoding floats as ints if conversion
> takes place without loss. This is a straightforward approach, but to
> implement it we need to care about reversed (decoding) situation.
The bug is confirmed: https://github.com/tarantool/tarantool/issues/3907
I agree with Kostja - it is just a bug, that Vinyl treats differently
integers and their double casts. It should not affect design decisions
of this patchset.
But there is another reason why we can't pass *.0 as an iterator value -
our fast comparators (TupleCompare, TupleCompareWithKey) are designed to
work with only values of same MP_ type. They do not use slow
tuple_compare_field() which is able to compare double and integer.
Moreover, I think, we should forbid implicit *.0 -> * This is why we
designed strict typing, isn't it?
If a table has column int, we should not be able to select it by double
values without explicit cast, even if implicit is possible without loss.
For explicit cast users have CAST operator.
What do you think?
>
> OP_Column fetches from msgpack field with given number and stores it as
> a native VDBE memory object. Type of that memory is based on type of
> msgpack value. So, if space field is of type NUMBER and holds value 1,
> type of VDBE memory will be INT (after decoding), not float 1.0. As a
> result, further calculations may be wrong: for instance, instead of
> floating point division, we could get integer division. To cope with
> this problem, lets add auxiliary conversion to decoding routine which
> uses space format of tuple to be decoded. It is worth mentioning that
> ephemeral spaces don't feature space format, so we are going to rely on
> type of key parts. Finally, internal VDBE merge sorter also operates on
> entries encoded into msgpack. To fix this case, we check type of
> ORDER BY/GROUP BY arguments: if they are of type float, we are emitting
> additional opcode OP_AffinityReal to force float type after encoding.
>
> Part of #3698
> ---
> src/box/field_def.h | 1 -
> src/box/lua/lua_sql.c | 2 +-
> src/box/sql.c | 10 +++++++---
> src/box/sql/build.c | 1 -
> src/box/sql/expr.c | 20 +++++++++++---------
> src/box/sql/select.c | 10 +++++++---
> src/box/sql/sqliteInt.h | 2 +-
> src/box/sql/vdbe.c | 26 ++++++++++++++++++--------
> src/box/sql/vdbeaux.c | 19 ++++++++++++++++---
> src/box/sql/vdbemem.c | 7 ++-----
> test/sql-tap/tkt-80e031a00f.test.lua | 12 ++++++------
> 11 files changed, 69 insertions(+), 41 deletions(-)
>
> diff --git a/src/box/sql.c b/src/box/sql.c
> index a06c50dca..a498cd8fe 100644
> --- a/src/box/sql.c
> +++ b/src/box/sql.c
> @@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
> for (uint32_t i = 0; i < field_count; ++i) {
> struct key_part_def *part = &ephemer_key_parts[i];
> part->fieldno = i;
> - part->type = FIELD_TYPE_SCALAR;
> part->nullable_action = ON_CONFLICT_ACTION_NONE;
> part->is_nullable = true;
> part->sort_order = SORT_ORDER_ASC;
> - if (def != NULL && i < def->part_count)
> + if (def != NULL && i < def->part_count) {
> + assert(def->parts[i].type < field_type_MAX);
> + part->type = def->parts[i].type != FIELD_TYPE_ANY ?
> + def->parts[i].type : FIELD_TYPE_SCALAR;
> part->coll_id = def->parts[i].coll_id;
1. How can key_part have FIELD_TYPE_ANY? We have no comparators for ANY
type, it is impossible, isn't it?
> - else
> + } else {
> part->coll_id = COLL_NONE;
> + part->type = FIELD_TYPE_SCALAR;
> + }
> }
> struct key_def *ephemer_key_def = key_def_new(ephemer_key_parts,
> field_count);
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index b67b22c23..7a0b929a7 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -3700,6 +3698,10 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
> sqlite3VdbeAddOp3(v, OP_Column,
> pAggInfo->sortingIdxPTab,
> pCol->iSorterColumn, target);
> + if (pCol->space_def->fields[pExpr->iAgg].type ==
> + FIELD_TYPE_NUMBER)
> + sqlite3VdbeAddOp1(v, OP_RealAffinity,
> + target);
2. Please, use {} for multi-line 'if' body.
> return target;
> }
> /* Otherwise, fall thru into the TK_COLUMN case */
> diff --git a/src/box/sql/select.c b/src/box/sql/select.c
> index 02ee225f1..40336e679 100644
> --- a/src/box/sql/select.c
> +++ b/src/box/sql/select.c
> @@ -1427,6 +1427,8 @@ sql_expr_list_to_key_info(struct Parse *parse, struct ExprList *list, int start)
> sql_expr_coll(parse, item->pExpr, &unused, &id);
> part->coll_id = id;
> part->sort_order = item->sort_order;
> + enum affinity_type aff = sqlite3ExprAffinity(item->pExpr);
> + part->type =sql_affinity_to_field_type(aff);
3. Lost white-space after '='.
> }
> return key_info;
> }
> @@ -5789,8 +5790,11 @@ sqlite3Select(Parse * pParse, /* The parser context */
> /* If the output is destined for a temporary table, open that table.
> */
> if (pDest->eDest == SRT_EphemTab) {
> - sqlite3VdbeAddOp2(v, OP_OpenTEphemeral, pDest->reg_eph,
> - pEList->nExpr + 1);
> + struct sql_key_info *key_info =
> + sql_expr_list_to_key_info(pParse, pEList, 0);
> + sqlite3VdbeAddOp4(v, OP_OpenTEphemeral, pDest->reg_eph,
> + pEList->nExpr + 1, 0, (char *)key_info,
> + P4_KEYINFO);
> sqlite3VdbeAddOp3(v, OP_IteratorOpen, pDest->iSDParm, 0,
> pDest->reg_eph);
4. Why can some ephemeral spaces be created without key_info? I grepped
by OP_OpenTEphemeral and saw that somewhere it is still Op2, not Op4.
>
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index e6b413c70..4345af24e 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -2173,7 +2167,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
> } else {
> /* Neither operand is NULL. Do a comparison. */
> affinity = pOp->p5 & AFFINITY_MASK;
> - if (affinity>=AFFINITY_NUMERIC) {
> + if (affinity>=AFFINITY_INTEGER) {
5. I guess, we have sqlite3IsNumericAffinity for this.
> if ((flags1 | flags3)&MEM_Str) {
> if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
> applyNumericAffinity(pIn1,0);
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2018-12-29 17:42 ` Vladislav Shpilevoy
@ 2019-01-09 8:26 ` Konstantin Osipov
2019-01-16 14:26 ` n.pettik
1 sibling, 0 replies; 48+ messages in thread
From: Konstantin Osipov @ 2019-01-09 8:26 UTC (permalink / raw)
To: tarantool-patches; +Cc: Nikita Pettik
* Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [18/12/29 20:46]:
> But there is another reason why we can't pass *.0 as an iterator value -
> our fast comparators (TupleCompare, TupleCompareWithKey) are designed to
> work with only values of same MP_ type. They do not use slow
> tuple_compare_field() which is able to compare double and integer.
If column data type is REAL, *.0 should be allowed. If column data
type is INT, *.0 should be either forbidden or implicitly cast to
INT (afaiu we chose to allow some implicit casts). But this cast,
if it is allowed, should happen before the value reaches the
iterator.
> Moreover, I think, we should forbid implicit *.0 -> * This is why we
> designed strict typing, isn't it?
Nikita has created a table of implicit cast behaviour by vendor,
and for all cases not explicitly prescribed by the standard we
will have to follow the majority of vendors. If there is no
majority, we'll stick with Oracle behaviour.
> If a table has column int, we should not be able to select it by double
> values without explicit cast, even if implicit is possible without loss.
> For explicit cast users have CAST operator.
>
> What do you think?
It's not a question of opinion, it's a question of policy.
Our attitude to casts is:
1. Follow the standard.
2. Follow the majority.
3. Follow Oracle.
--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-09 8:26 ` Konstantin Osipov
@ 2019-01-16 14:26 ` n.pettik
2019-01-22 15:41 ` Vladislav Shpilevoy
1 sibling, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-16 14:26 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> Numeric affinity in SQLite means the same as real, except that it
>> forces integer values into floating point representation in case
>> it can be converted without loss (e.g. 2.0 -> 2).
>> Since in Tarantool core there is no difference between numeric and real
>> values (both are stored as values of type NUMBER), lets remove numeric
>> affinity and use instead real.
>> The only real pitfall is implicit conversion mentioned above.
>> What is more, vinyl engine complicates problem since it relies
>> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
>> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
>> be able to use iterators from Lua, since they implicitly change type of
>> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
>> simple: lets always attempt at encoding floats as ints if conversion
>> takes place without loss. This is a straightforward approach, but to
>> implement it we need to care about reversed (decoding) situation.
>
> The bug is confirmed: https://github.com/tarantool/tarantool/issues/3907
>
> I agree with Kostja - it is just a bug, that Vinyl treats differently
> integers and their double casts. It should not affect design decisions
> of this patchset.
Ok, we may consider part of this patch as a workaround.
Now affinity removal would significantly help me to fix
other issues connected with strict typing (e.g. implicit casts).
afterwards code introduced in this commit may be simplified/removed.
I don’t know what priority of #3907 issue (milestone is 2.1.1 but
we know that sometimes it may take a while).
> But there is another reason why we can't pass *.0 as an iterator value -
> our fast comparators (TupleCompare, TupleCompareWithKey) are designed to
> work with only values of same MP_ type. They do not use slow
> tuple_compare_field() which is able to compare double and integer.
Yep, it is sad. I can’t say anything more now,I need to "think about it”.
The only workaround I can come up right now with is to add to
every OP_Found/OP_Seek etc opcode OP_PromoteType.
It would attempt at fetching space from cursor and gently
applying types from format to record to be passed to iterator.
But this doesn’t seem to be acceptable solution for many reasons.
Otherwise, I have no idea how to determine required conversions
*.0 -> * without execution context.
> Moreover, I think, we should forbid implicit *.0 -> * This is why we
> designed strict typing, isn't it?
It is too strict rule, ANSI allows this kind of conversion.
The only restriction is that there shouldn’t be precision loss
during conversion. It has been already discussed with
Konstantin and Peter Gulutzan, I inline part of that thread
(Topic “[dev] Re: Casts")
> The standard rules for casting look like this
> (where SD = source data type, TD = target data type, SV = source value):
> "
> 8) If TD is exact numeric, then
> Case:
> a)If SD is exact numeric or approximate numeric, then
> Case:
> i) If there is a representation of SV in the data type TD that does
> not lose any leading significant digits after rounding or truncating
> if necessary, then TV is that representation.
> The choice of whether to round or truncate is implementation-defined.
> ii)Otherwise, an exception condition is raised:
> data exception — numeric value out of range.
> "
> So truncating 1.123 to 1 is okay, but truncating 11.123 to 1.123 is not okay
> (that loses a leading significant digit).
>
Even more simple situation:
CREATE TABLE t1 (id INT PRIMARY KEY);
INSERT INTO t1 VALUES (1.0);
This example is valid and all DBs support it.
> If a table has column int, we should not be able to select it by double
> values without explicit cast, even if implicit is possible without loss.
> For explicit cast users have CAST operator.
It would look at least strange:
SELECT * FROM t1 WHERE x = (1.0 AS INT);
Again, all DBs I know accept this query without cast.
>
>> OP_Column fetches from msgpack field with given number and stores it as
>> a native VDBE memory object. Type of that memory is based on type of
>> msgpack value. So, if space field is of type NUMBER and holds value 1,
>> type of VDBE memory will be INT (after decoding), not float 1.0. As a
>> result, further calculations may be wrong: for instance, instead of
>> floating point division, we could get integer division. To cope with
>> this problem, lets add auxiliary conversion to decoding routine which
>> uses space format of tuple to be decoded. It is worth mentioning that
>> ephemeral spaces don't feature space format, so we are going to rely on
>> type of key parts. Finally, internal VDBE merge sorter also operates on
>> entries encoded into msgpack. To fix this case, we check type of
>> ORDER BY/GROUP BY arguments: if they are of type float, we are emitting
>> additional opcode OP_AffinityReal to force float type after encoding.
>> Part of #3698
>> ---
>> src/box/field_def.h | 1 -
>> src/box/lua/lua_sql.c | 2 +-
>> src/box/sql.c | 10 +++++++---
>> src/box/sql/build.c | 1 -
>> src/box/sql/expr.c | 20 +++++++++++---------
>> src/box/sql/select.c | 10 +++++++---
>> src/box/sql/sqliteInt.h | 2 +-
>> src/box/sql/vdbe.c | 26 ++++++++++++++++++--------
>> src/box/sql/vdbeaux.c | 19 ++++++++++++++++---
>> src/box/sql/vdbemem.c | 7 ++-----
>> test/sql-tap/tkt-80e031a00f.test.lua | 12 ++++++------
>> 11 files changed, 69 insertions(+), 41 deletions(-)
>> diff --git a/src/box/sql.c b/src/box/sql.c
>> index a06c50dca..a498cd8fe 100644
>> --- a/src/box/sql.c
>> +++ b/src/box/sql.c
>> @@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
>> for (uint32_t i = 0; i < field_count; ++i) {
>> struct key_part_def *part = &ephemer_key_parts[i];
>> part->fieldno = i;
>> - part->type = FIELD_TYPE_SCALAR;
>> part->nullable_action = ON_CONFLICT_ACTION_NONE;
>> part->is_nullable = true;
>> part->sort_order = SORT_ORDER_ASC;
>> - if (def != NULL && i < def->part_count)
>> + if (def != NULL && i < def->part_count) {
>> + assert(def->parts[i].type < field_type_MAX);
>> + part->type = def->parts[i].type != FIELD_TYPE_ANY ?
>> + def->parts[i].type : FIELD_TYPE_SCALAR;
>> part->coll_id = def->parts[i].coll_id;
>
> 1. How can key_part have FIELD_TYPE_ANY? We have no comparators for ANY
> type, it is impossible, isn't it?
We don’t, and that is why I replace ANY with SCALAR.
>
>> - else
>> + } else {
>> part->coll_id = COLL_NONE;
>> + part->type = FIELD_TYPE_SCALAR;
>> + }
>> }
>> struct key_def *ephemer_key_def = key_def_new(ephemer_key_parts,
>> field_count);
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index b67b22c23..7a0b929a7 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -3700,6 +3698,10 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
>> sqlite3VdbeAddOp3(v, OP_Column,
>> pAggInfo->sortingIdxPTab,
>> pCol->iSorterColumn, target);
>> + if (pCol->space_def->fields[pExpr->iAgg].type ==
>> + FIELD_TYPE_NUMBER)
>> + sqlite3VdbeAddOp1(v, OP_RealAffinity,
>> + target);
>
> 2. Please, use {} for multi-line 'if' body.
Ok (I always confuse multi-line with multi-statement body):
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 7a0b929a7..14b727039 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3699,9 +3699,10 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
pAggInfo->sortingIdxPTab,
pCol->iSorterColumn, target);
if (pCol->space_def->fields[pExpr->iAgg].type ==
- FIELD_TYPE_NUMBER)
+ FIELD_TYPE_NUMBER) {
sqlite3VdbeAddOp1(v, OP_RealAffinity,
target);
+ }
return target;
}
>
>> return target;
>> }
>> /* Otherwise, fall thru into the TK_COLUMN case */
>> diff --git a/src/box/sql/select.c b/src/box/sql/select.c
>> index 02ee225f1..40336e679 100644
>> --- a/src/box/sql/select.c
>> +++ b/src/box/sql/select.c
>> @@ -1427,6 +1427,8 @@ sql_expr_list_to_key_info(struct Parse *parse, struct ExprList *list, int start)
>> sql_expr_coll(parse, item->pExpr, &unused, &id);
>> part->coll_id = id;
>> part->sort_order = item->sort_order;
>> + enum affinity_type aff = sqlite3ExprAffinity(item->pExpr);
>> + part->type =sql_affinity_to_field_type(aff);
>
> 3. Lost white-space after '=‘.
Fixed:
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 40336e679..fa0fea6c8 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1428,7 +1428,7 @@ sql_expr_list_to_key_info(struct Parse *parse, struct ExprList *list, int start)
part->coll_id = id;
part->sort_order = item->sort_order;
enum affinity_type aff = sqlite3ExprAffinity(item->pExpr);
- part->type =sql_affinity_to_field_type(aff);
+ part->type = sql_affinity_to_field_type(aff);
>
>> }
>> return key_info;
>> }
>> @@ -5789,8 +5790,11 @@ sqlite3Select(Parse * pParse, /* The parser context */
>> /* If the output is destined for a temporary table, open that table.
>> */
>> if (pDest->eDest == SRT_EphemTab) {
>> - sqlite3VdbeAddOp2(v, OP_OpenTEphemeral, pDest->reg_eph,
>> - pEList->nExpr + 1);
>> + struct sql_key_info *key_info =
>> + sql_expr_list_to_key_info(pParse, pEList, 0);
>> + sqlite3VdbeAddOp4(v, OP_OpenTEphemeral, pDest->reg_eph,
>> + pEList->nExpr + 1, 0, (char *)key_info,
>> + P4_KEYINFO);
>> sqlite3VdbeAddOp3(v, OP_IteratorOpen, pDest->iSDParm, 0,
>> pDest->reg_eph);
>
> 4. Why can some ephemeral spaces be created without key_info? I grepped
> by OP_OpenTEphemeral and saw that somewhere it is still Op2, not Op4.
If there is no key_info, then default one is used instead.
See sql_ephemeral_space_create and how we cope
with situation when key_info == NULL.
>
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index e6b413c70..4345af24e 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -2173,7 +2167,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
>> } else {
>> /* Neither operand is NULL. Do a comparison. */
>> affinity = pOp->p5 & AFFINITY_MASK;
>> - if (affinity>=AFFINITY_NUMERIC) {
>> + if (affinity>=AFFINITY_INTEGER) {
>
> 5. I guess, we have sqlite3IsNumericAffinity for this.
Anyway I replace it with sql_affinity_is_numeric in
sql: clean-up affinity from SQL source code patch.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2019-01-16 14:26 ` n.pettik
@ 2019-01-22 15:41 ` Vladislav Shpilevoy
2019-01-28 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-22 15:41 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Hi! Thanks for the fixes!
On 16/01/2019 17:26, n.pettik wrote:
>
>>> Numeric affinity in SQLite means the same as real, except that it
>>> forces integer values into floating point representation in case
>>> it can be converted without loss (e.g. 2.0 -> 2).
>>> Since in Tarantool core there is no difference between numeric and real
>>> values (both are stored as values of type NUMBER), lets remove numeric
>>> affinity and use instead real.
>>> The only real pitfall is implicit conversion mentioned above.
>>> What is more, vinyl engine complicates problem since it relies
>>> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
>>> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
>>> be able to use iterators from Lua, since they implicitly change type of
>>> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
>>> simple: lets always attempt at encoding floats as ints if conversion
>>> takes place without loss. This is a straightforward approach, but to
>>> implement it we need to care about reversed (decoding) situation.
>>
>> The bug is confirmed: https://github.com/tarantool/tarantool/issues/3907
>>
>> I agree with Kostja - it is just a bug, that Vinyl treats differently
>> integers and their double casts. It should not affect design decisions
>> of this patchset.
>
> Ok, we may consider part of this patch as a workaround.
> Now affinity removal would significantly help me to fix
> other issues connected with strict typing (e.g. implicit casts).
> afterwards code introduced in this commit may be simplified/removed.
> I don’t know what priority of #3907 issue (milestone is 2.1.1 but
> we know that sometimes it may take a while).
As we agreed, it is just a Vinyl bug, and now this part of the
commit message looks strange:
"
The only real pitfall is implicit conversion mentioned above.
What is more, vinyl engine complicates problem since it relies
on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
be able to use iterators from Lua, since they implicitly change type of
1.0 and pass it to the iterator as MP_INT.
"
Here you've stated that Vinyl even for number indexes sees a difference
between 2.0 and 2, but it is wrong (in an ideal world, but in our it is
just a bug). It is better to write here about not a temporary bug, but
about, for instance, the example I've showed you below.
>
>> But there is another reason why we can't pass *.0 as an iterator value -
>> our fast comparators (TupleCompare, TupleCompareWithKey) are designed to
>> work with only values of same MP_ type. They do not use slow
>> tuple_compare_field() which is able to compare double and integer.
>
> Yep, it is sad. I can’t say anything more now,I need to "think about it”.
Please, use this example or find another to support your decision
always 'integerifying' float numbers having zero fraction.
>
> The only workaround I can come up right now with is to add to
> every OP_Found/OP_Seek etc opcode OP_PromoteType.
> It would attempt at fetching space from cursor and gently
> applying types from format to record to be passed to iterator.
> But this doesn’t seem to be acceptable solution for many reasons.
I think, the implementation in this patch is ok. It is not
necessary to create new opcodes, nor think up another solution.
>
> Otherwise, I have no idea how to determine required conversions
> *.0 -> * without execution context.
It is impossible of course. You can not learn what a value you
will fetch from cursors before VDBE execution.
>
>> Moreover, I think, we should forbid implicit *.0 -> * This is why we
>> designed strict typing, isn't it?
>
> It is too strict rule, ANSI allows this kind of conversion.
> The only restriction is that there shouldn’t be precision loss
> during conversion. It has been already discussed with
> Konstantin and Peter Gulutzan, I inline part of that thread
> (Topic “[dev] Re: Casts")
Understood, ok. Thanks for the investigation.
>>> diff --git a/src/box/sql.c b/src/box/sql.c
>>> index a06c50dca..a498cd8fe 100644
>>> --- a/src/box/sql.c
>>> +++ b/src/box/sql.c
>>> @@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
>>> for (uint32_t i = 0; i < field_count; ++i) {
>>> struct key_part_def *part = &ephemer_key_parts[i];
>>> part->fieldno = i;
>>> - part->type = FIELD_TYPE_SCALAR;
>>> part->nullable_action = ON_CONFLICT_ACTION_NONE;
>>> part->is_nullable = true;
>>> part->sort_order = SORT_ORDER_ASC;
>>> - if (def != NULL && i < def->part_count)
>>> + if (def != NULL && i < def->part_count) {
>>> + assert(def->parts[i].type < field_type_MAX);
>>> + part->type = def->parts[i].type != FIELD_TYPE_ANY ?
>>> + def->parts[i].type : FIELD_TYPE_SCALAR;
>>> part->coll_id = def->parts[i].coll_id;
>>
>> 1. How can key_part have FIELD_TYPE_ANY? We have no comparators for ANY
>> type, it is impossible, isn't it?
>
> We don’t, and that is why I replace ANY with SCALAR.
No, you still check for "def->parts[i].type != FIELD_TYPE_ANY", and I
can not understand how is it possible. struct key_def can not have
FIELD_TYPE_ANY in its parts.
>
>>
>>> - else
>>> + } else {
>>> part->coll_id = COLL_NONE;
>>> + part->type = FIELD_TYPE_SCALAR;
>>> + }
>>> }
>>> struct key_def *ephemer_key_def = key_def_new(ephemer_key_parts,
>>> field_count);
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2019-01-22 15:41 ` Vladislav Shpilevoy
@ 2019-01-28 16:39 ` n.pettik
2019-01-30 13:04 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-28 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
[-- Attachment #1: Type: text/plain, Size: 4477 bytes --]
> On 16/01/2019 17:26, n.pettik wrote:
>>>> Numeric affinity in SQLite means the same as real, except that it
>>>> forces integer values into floating point representation in case
>>>> it can be converted without loss (e.g. 2.0 -> 2).
>>>> Since in Tarantool core there is no difference between numeric and real
>>>> values (both are stored as values of type NUMBER), lets remove numeric
>>>> affinity and use instead real.
>>>> The only real pitfall is implicit conversion mentioned above.
>>>> What is more, vinyl engine complicates problem since it relies
>>>> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
>>>> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
>>>> be able to use iterators from Lua, since they implicitly change type of
>>>> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
>>>> simple: lets always attempt at encoding floats as ints if conversion
>>>> takes place without loss. This is a straightforward approach, but to
>>>> implement it we need to care about reversed (decoding) situation.
>>>
>>> The bug is confirmed: https://github.com/tarantool/tarantool/issues/3907
>>>
>>> I agree with Kostja - it is just a bug, that Vinyl treats differently
>>> integers and their double casts. It should not affect design decisions
>>> of this patchset.
>> Ok, we may consider part of this patch as a workaround.
>> Now affinity removal would significantly help me to fix
>> other issues connected with strict typing (e.g. implicit casts).
>> afterwards code introduced in this commit may be simplified/removed.
>> I don’t know what priority of #3907 issue (milestone is 2.1.1 but
>> we know that sometimes it may take a while).
>
> As we agreed, it is just a Vinyl bug, and now this part of the
> commit message looks strange:
>
> "
> The only real pitfall is implicit conversion mentioned above.
> What is more, vinyl engine complicates problem since it relies
> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
> be able to use iterators from Lua, since they implicitly change type of
> 1.0 and pass it to the iterator as MP_INT.
> "
>
> Here you've stated that Vinyl even for number indexes sees a difference
> between 2.0 and 2, but it is wrong (in an ideal world, but in our it is
> just a bug). It is better to write here about not a temporary bug, but
> about, for instance, the example I've showed you below.
>
>>> But there is another reason why we can't pass *.0 as an iterator value -
>>> our fast comparators (TupleCompare, TupleCompareWithKey) are designed to
>>> work with only values of same MP_ type. They do not use slow
>>> tuple_compare_field() which is able to compare double and integer.
>> Yep, it is sad. I can’t say anything more now,I need to "think about it”.
>
> Please, use this example or find another to support your decision
> always 'integerifying' float numbers having zero fraction.
Ok, I replaced original paragraph with yours.
>>>> diff --git a/src/box/sql.c b/src/box/sql.c
>>>> index a06c50dca..a498cd8fe 100644
>>>> --- a/src/box/sql.c
>>>> +++ b/src/box/sql.c
>>>> @@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
>>>> for (uint32_t i = 0; i < field_count; ++i) {
>>>> struct key_part_def *part = &ephemer_key_parts[i];
>>>> part->fieldno = i;
>>>> - part->type = FIELD_TYPE_SCALAR;
>>>> part->nullable_action = ON_CONFLICT_ACTION_NONE;
>>>> part->is_nullable = true;
>>>> part->sort_order = SORT_ORDER_ASC;
>>>> - if (def != NULL && i < def->part_count)
>>>> + if (def != NULL && i < def->part_count) {
>>>> + assert(def->parts[i].type < field_type_MAX);
>>>> + part->type = def->parts[i].type != FIELD_TYPE_ANY ?
>>>> + def->parts[i].type : FIELD_TYPE_SCALAR;
>>>> part->coll_id = def->parts[i].coll_id;
>>>
>>> 1. How can key_part have FIELD_TYPE_ANY? We have no comparators for ANY
>>> type, it is impossible, isn't it?
>> We don’t, and that is why I replace ANY with SCALAR.
>
> No, you still check for "def->parts[i].type != FIELD_TYPE_ANY", and I
> can not understand how is it possible. struct key_def can not have
> FIELD_TYPE_ANY in its parts.
Now this problem is fixed in the next patches.
In this one it can’t be fixed with ease.
[-- Attachment #2: Type: text/html, Size: 24909 bytes --]
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2019-01-28 16:39 ` n.pettik
@ 2019-01-30 13:04 ` Vladislav Shpilevoy
2019-02-01 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-30 13:04 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Hi! See a comment below.
On 28/01/2019 19:39, n.pettik wrote:
>
>> On 16/01/2019 17:26, n.pettik wrote:
>>>>> Numeric affinity in SQLite means the same as real, except that it
>>>>> forces integer values into floating point representation in case
>>>>> it can be converted without loss (e.g. 2.0 -> 2).
>>>>> Since in Tarantool core there is no difference between numeric and real
>>>>> values (both are stored as values of type NUMBER), lets remove numeric
>>>>> affinity and use instead real.
>>>>> The only real pitfall is implicit conversion mentioned above.
>>>>> What is more, vinyl engine complicates problem since it relies
>>>>> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
>>>>> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
>>>>> be able to use iterators from Lua, since they implicitly change type of
>>>>> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
>>>>> simple: lets always attempt at encoding floats as ints if conversion
>>>>> takes place without loss. This is a straightforward approach, but to
>>>>> implement it we need to care about reversed (decoding) situation.
>>>>
>>>> The bug is confirmed: https://github.com/tarantool/tarantool/issues/3907
>>>>
>>>> I agree with Kostja - it is just a bug, that Vinyl treats differently
>>>> integers and their double casts. It should not affect design decisions
>>>> of this patchset.
>>> Ok, we may consider part of this patch as a workaround.
>>> Now affinity removal would significantly help me to fix
>>> other issues connected with strict typing (e.g. implicit casts).
>>> afterwards code introduced in this commit may be simplified/removed.
>>> I don’t know what priority of #3907 issue (milestone is 2.1.1 but
>>> we know that sometimes it may take a while).
>>
>> As we agreed, it is just a Vinyl bug, and now this part of the
>> commit message looks strange:
>>
>> "
>> The only real pitfall is implicit conversion mentioned above.
>> What is more, vinyl engine complicates problem since it relies
>> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
>> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
>> be able to use iterators from Lua, since they implicitly change type of
>> 1.0 and pass it to the iterator as MP_INT.
>> "
>>
>> Here you've stated that Vinyl even for number indexes sees a difference
>> between 2.0 and 2, but it is wrong (in an ideal world, but in our it is
>> just a bug). It is better to write here about not a temporary bug, but
>> about, for instance, the example I've showed you below.
>>
>>>> But there is another reason why we can't pass *.0 as an iterator value -
>>>> our fast comparators (TupleCompare, TupleCompareWithKey) are designed to
>>>> work with only values of same MP_ type. They do not use slow
>>>> tuple_compare_field() which is able to compare double and integer.
>>> Yep, it is sad. I can’t say anything more now,I need to "think about it”.
>>
>> Please, use this example or find another to support your decision
>> always 'integerifying' float numbers having zero fraction.
>
> Ok, I replaced original paragraph with yours.
>
>>>>> diff --git a/src/box/sql.c b/src/box/sql.c
>>>>> index a06c50dca..a498cd8fe 100644
>>>>> --- a/src/box/sql.c
>>>>> +++ b/src/box/sql.c
>>>>> @@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
>>>>> for (uint32_t i = 0; i < field_count; ++i) {
>>>>> struct key_part_def *part = &ephemer_key_parts[i];
>>>>> part->fieldno = i;
>>>>> -part->type = FIELD_TYPE_SCALAR;
>>>>> part->nullable_action = ON_CONFLICT_ACTION_NONE;
>>>>> part->is_nullable = true;
>>>>> part->sort_order = SORT_ORDER_ASC;
>>>>> -if (def != NULL && i < def->part_count)
>>>>> +if (def != NULL && i < def->part_count) {
>>>>> +assert(def->parts[i].type < field_type_MAX);
>>>>> +part->type = def->parts[i].type != FIELD_TYPE_ANY ?
>>>>> + def->parts[i].type : FIELD_TYPE_SCALAR;
>>>>> part->coll_id = def->parts[i].coll_id;
>>>>
>>>> 1. How can key_part have FIELD_TYPE_ANY? We have no comparators for ANY
>>>> type, it is impossible, isn't it?
>>> We don’t, and that is why I replace ANY with SCALAR.
>>
>> No, you still check for "def->parts[i].type != FIELD_TYPE_ANY", and I
>> can not understand how is it possible. struct key_def can not have
>> FIELD_TYPE_ANY in its parts.
>
> Now this problem is fixed in the next patches.
> In this one it can’t be fixed with ease.
>
What a problem can not be fixed? I did this:
diff --git a/src/box/sql.c b/src/box/sql.c
index 530ec2384..f53600837 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -387,8 +387,7 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
part->sort_order = SORT_ORDER_ASC;
if (def != NULL && i < def->part_count) {
assert(def->parts[i].type < field_type_MAX);
- part->type = def->parts[i].type != FIELD_TYPE_ANY ?
- def->parts[i].type : FIELD_TYPE_SCALAR;
+ part->type = def->parts[i].type;
part->coll_id = def->parts[i].coll_id;
} else {
part->coll_id = COLL_NONE;
And no test failed. This is because, as I said, key_part.type
is never ever can be ANY. I was not talking above about ANY
removal from all places, but from this particular one. And here
we can be sure, that key_part.type != FIELD_TYPE_ANY always.
Please, apply this.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2019-01-30 13:04 ` Vladislav Shpilevoy
@ 2019-02-01 16:39 ` n.pettik
0 siblings, 0 replies; 48+ messages in thread
From: n.pettik @ 2019-02-01 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>>>>> diff --git a/src/box/sql.c b/src/box/sql.c
>>>>>> index a06c50dca..a498cd8fe 100644
>>>>>> --- a/src/box/sql.c
>>>>>> +++ b/src/box/sql.c
>>>>>> @@ -376,14 +376,18 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
>>>>>> for (uint32_t i = 0; i < field_count; ++i) {
>>>>>> struct key_part_def *part = &ephemer_key_parts[i];
>>>>>> part->fieldno = i;
>>>>>> -part->type = FIELD_TYPE_SCALAR;
>>>>>> part->nullable_action = ON_CONFLICT_ACTION_NONE;
>>>>>> part->is_nullable = true;
>>>>>> part->sort_order = SORT_ORDER_ASC;
>>>>>> -if (def != NULL && i < def->part_count)
>>>>>> +if (def != NULL && i < def->part_count) {
>>>>>> +assert(def->parts[i].type < field_type_MAX);
>>>>>> +part->type = def->parts[i].type != FIELD_TYPE_ANY ?
>>>>>> + def->parts[i].type : FIELD_TYPE_SCALAR;
>>>>>> part->coll_id = def->parts[i].coll_id;
>>>>>
>>>>> 1. How can key_part have FIELD_TYPE_ANY? We have no comparators for ANY
>>>>> type, it is impossible, isn't it?
>>>> We don’t, and that is why I replace ANY with SCALAR.
>>>
>>> No, you still check for "def->parts[i].type != FIELD_TYPE_ANY", and I
>>> can not understand how is it possible. struct key_def can not have
>>> FIELD_TYPE_ANY in its parts.
>> Now this problem is fixed in the next patches.
>> In this one it can’t be fixed with ease.
>
> What a problem can not be fixed? I did this:
>
> diff --git a/src/box/sql.c b/src/box/sql.c
> index 530ec2384..f53600837 100644
> --- a/src/box/sql.c
> +++ b/src/box/sql.c
> @@ -387,8 +387,7 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
> part->sort_order = SORT_ORDER_ASC;
> if (def != NULL && i < def->part_count) {
> assert(def->parts[i].type < field_type_MAX);
> - part->type = def->parts[i].type != FIELD_TYPE_ANY ?
> - def->parts[i].type : FIELD_TYPE_SCALAR;
> + part->type = def->parts[i].type;
> part->coll_id = def->parts[i].coll_id;
> } else {
> part->coll_id = COLL_NONE;
>
> And no test failed. This is because, as I said, key_part.type
> is never ever can be ANY. I was not talking above about ANY
> removal from all places, but from this particular one. And here
> we can be sure, that key_part.type != FIELD_TYPE_ANY always.
>
> Please, apply this.
Ok. I've understood that it can't be of type ANY due to
the fact that sql_affinity_to_field_type (which is used to init sql_key_info)
never returns ANY and default value is SCALAR.
Applied.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] Re: [PATCH 3/8] sql: remove numeric affinity
2018-12-28 9:34 ` [tarantool-patches] [PATCH 3/8] sql: remove numeric affinity Nikita Pettik
2018-12-29 9:01 ` [tarantool-patches] " Konstantin Osipov
2018-12-29 17:42 ` Vladislav Shpilevoy
@ 2019-01-09 8:20 ` Konstantin Osipov
2 siblings, 0 replies; 48+ messages in thread
From: Konstantin Osipov @ 2019-01-09 8:20 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
* Nikita Pettik <korablev@tarantool.org> [18/12/28 12:36]:
> Numeric affinity in SQLite means the same as real, except that it
> forces integer values into floating point representation in case
> it can be converted without loss (e.g. 2.0 -> 2).
> Since in Tarantool core there is no difference between numeric and real
> values (both are stored as values of type NUMBER), lets remove numeric
> affinity and use instead real.
>
> The only real pitfall is implicit conversion mentioned above.
> What is more, vinyl engine complicates problem since it relies
> on data encoding (i.e. whether it is encoded as MP_INT or MP_FLOAT).
> For instance, if we encode 1.0 as MP_FLOAT during insertion, we won't
> be able to use iterators from Lua, since they implicitly change type of
> 1.0 and pass it to the iterator as MP_INT. Solution to this problem is
> simple: lets always attempt at encoding floats as ints if conversion
> takes place without loss. This is a straightforward approach, but to
> implement it we need to care about reversed (decoding) situation.
Nikita, you're mixing up msgpack formats, msgpack types, and
tarantool types.
You're also putting some unjust blame on vinyl, while, albeit
there is a problem in vinyl, it's the responsibility of SQL layer
to encode and decode data properly.
> OP_Column fetches from msgpack field with given number and stores it as
> a native VDBE memory object. Type of that memory is based on type of
> msgpack value. So, if space field is of type NUMBER and holds value 1,
> type of VDBE memory will be INT (after decoding), not float 1.0.
NUMBER is not a msgpack type, it's a tarantool type.
Msgpack data types (check out on wikipedia or in the spec) are
int, bool, float, str, bin, array, map and ext.
Tarantool data types, check out in the manual, are integer,
string, number.
We're not using msgpack data types - generally they should be
irrelevant, except perhaps on the client side, when data is
received over the binary protocol.
--
Konstantin Osipov, Moscow, Russia, +7 903 626 22 32
http://tarantool.io - www.twitter.com/kostja_osipov
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] [PATCH 4/8] sql: replace affinity with field type for func
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (2 preceding siblings ...)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 3/8] sql: remove numeric affinity Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-28 9:34 ` [tarantool-patches] [PATCH 5/8] sql: replace field type with affinity for VDBE runtime Nikita Pettik
` (5 subsequent siblings)
9 siblings, 0 replies; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Lets use field_type instead of affinity as a type of return value of
user function registered in SQL.
Part of #3698
---
src/box/lua/lua_sql.c | 12 +++----
src/box/sql/date.c | 17 ++++-----
src/box/sql/expr.c | 7 ++--
src/box/sql/func.c | 91 +++++++++++++++++++++++--------------------------
src/box/sql/main.c | 4 +--
src/box/sql/sqliteInt.h | 6 ++--
6 files changed, 67 insertions(+), 70 deletions(-)
diff --git a/src/box/lua/lua_sql.c b/src/box/lua/lua_sql.c
index ab0b7f37c..386da43df 100644
--- a/src/box/lua/lua_sql.c
+++ b/src/box/lua/lua_sql.c
@@ -149,18 +149,18 @@ lbox_sql_create_function(struct lua_State *L)
lua_isfunction(L, 3) && lua_isnumber(L, 4) &&
lua_isboolean(L, 5)))
return luaL_error(L, "Invalid arguments");
- enum affinity_type type = AFFINITY_UNDEFINED;
+ enum field_type type;
const char *type_arg = lua_tostring(L, 2);
if (strcmp(type_arg, "INT") == 0 || strcmp(type_arg, "INTEGER") == 0)
- type = AFFINITY_INTEGER;
+ type = FIELD_TYPE_INTEGER;
else if (strcmp(type_arg, "TEXT") == 0)
- type = AFFINITY_TEXT;
+ type = FIELD_TYPE_STRING;
else if (strcmp(type_arg, "FLOAT") == 0)
- type = AFFINITY_REAL;
+ type = FIELD_TYPE_NUMBER;
else if (strcmp(type_arg, "NUM") == 0)
- type = AFFINITY_REAL;
+ type = FIELD_TYPE_NUMBER;
else if (strcmp(type_arg, "BLOB") == 0)
- type = AFFINITY_BLOB;
+ type = FIELD_TYPE_SCALAR;
else
return luaL_error(L, "Unknown type");
/* -1 indicates any number of arguments. */
diff --git a/src/box/sql/date.c b/src/box/sql/date.c
index 8a3588355..e452aad73 100644
--- a/src/box/sql/date.c
+++ b/src/box/sql/date.c
@@ -1306,14 +1306,15 @@ sqlite3RegisterDateTimeFunctions(void)
{
static FuncDef aDateTimeFuncs[] = {
#ifndef SQLITE_OMIT_DATETIME_FUNCS
- DFUNCTION(julianday, -1, 0, 0, juliandayFunc, AFFINITY_REAL),
- DFUNCTION(date, -1, 0, 0, dateFunc, AFFINITY_REAL),
- DFUNCTION(time, -1, 0, 0, timeFunc, AFFINITY_REAL),
- DFUNCTION(datetime, -1, 0, 0, datetimeFunc, AFFINITY_REAL),
- DFUNCTION(strftime, -1, 0, 0, strftimeFunc, AFFINITY_REAL),
- DFUNCTION(current_time, 0, 0, 0, ctimeFunc, AFFINITY_REAL),
- DFUNCTION(current_timestamp, 0, 0, 0, ctimestampFunc, AFFINITY_REAL),
- DFUNCTION(current_date, 0, 0, 0, cdateFunc, AFFINITY_REAL),
+ DFUNCTION(julianday, -1, 0, 0, juliandayFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(date, -1, 0, 0, dateFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(time, -1, 0, 0, timeFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(datetime, -1, 0, 0, datetimeFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(strftime, -1, 0, 0, strftimeFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(current_time, 0, 0, 0, ctimeFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(current_timestamp, 0, 0, 0, ctimestampFunc,
+ FIELD_TYPE_NUMBER),
+ DFUNCTION(current_date, 0, 0, 0, cdateFunc, FIELD_TYPE_NUMBER),
#else
STR_FUNCTION(current_time, 0, "%H:%M:%S", 0, currentTimeFunc),
STR_FUNCTION(current_date, 0, "%Y-%m-%d", 0, currentTimeFunc),
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 7a0b929a7..917e6e30b 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3959,7 +3959,9 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
"misuse of aggregate: %s()",
pExpr->u.zToken);
} else {
- pExpr->affinity = pInfo->aFunc->pFunc->ret_type;
+ enum field_type t =
+ pInfo->aFunc->pFunc->ret_type;
+ pExpr->affinity = sql_field_type_to_affinity(t);
return pInfo->aFunc[pExpr->iAgg].iMem;
}
break;
@@ -3997,7 +3999,8 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
}
if (pDef->ret_type != AFFINITY_UNDEFINED) {
- pExpr->affinity = pDef->ret_type;
+ pExpr->affinity =
+ sql_field_type_to_affinity(pDef->ret_type);
} else {
/*
* Otherwise, use first arg as
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 9667aead5..7a2d3a77d 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1590,7 +1590,7 @@ groupConcatFinalize(sqlite3_context * context)
*/
static inline int
sqlite3_overload_function(sqlite3 * db, const char *zName,
- enum affinity_type type, int nArg)
+ enum field_type type, int nArg)
{
int rc = SQLITE_OK;
@@ -1615,7 +1615,7 @@ sqlite3_overload_function(sqlite3 * db, const char *zName,
void
sqlite3RegisterPerConnectionBuiltinFunctions(sqlite3 * db)
{
- int rc = sqlite3_overload_function(db, "MATCH", AFFINITY_UNDEFINED, 2);
+ int rc = sqlite3_overload_function(db, "MATCH", FIELD_TYPE_ANY, 2);
assert(rc == SQLITE_NOMEM || rc == SQLITE_OK);
if (rc == SQLITE_NOMEM) {
sqlite3OomFault(db);
@@ -1647,9 +1647,9 @@ sqlite3RegisterLikeFunctions(sqlite3 *db, int is_case_insensitive)
* supplied pattern and FALSE otherwise.
*/
int *is_like_ci = SQLITE_INT_TO_PTR(is_case_insensitive);
- sqlite3CreateFunc(db, "LIKE", AFFINITY_INTEGER, 2, 0,
+ sqlite3CreateFunc(db, "LIKE", FIELD_TYPE_INTEGER, 2, 0,
is_like_ci, likeFunc, 0, 0, 0);
- sqlite3CreateFunc(db, "LIKE", AFFINITY_INTEGER, 3, 0,
+ sqlite3CreateFunc(db, "LIKE", FIELD_TYPE_INTEGER, 3, 0,
is_like_ci, likeFunc, 0, 0, 0);
setLikeOptFlag(db, "LIKE",
!(is_case_insensitive) ? (SQLITE_FUNC_LIKE |
@@ -1704,17 +1704,17 @@ sqlite3RegisterBuiltinFunctions(void)
FUNCTION(soundex, 1, 0, 0, soundexFunc),
#endif
FUNCTION2(unlikely, 1, 0, 0, noopFunc, SQLITE_FUNC_UNLIKELY,
- AFFINITY_INTEGER),
+ FIELD_TYPE_INTEGER),
FUNCTION2(likelihood, 2, 0, 0, noopFunc, SQLITE_FUNC_UNLIKELY,
- AFFINITY_INTEGER),
+ FIELD_TYPE_INTEGER),
FUNCTION2(likely, 1, 0, 0, noopFunc, SQLITE_FUNC_UNLIKELY,
- AFFINITY_INTEGER),
- FUNCTION(ltrim, 1, 1, 0, trimFunc, AFFINITY_TEXT),
- FUNCTION(ltrim, 2, 1, 0, trimFunc, AFFINITY_TEXT),
- FUNCTION(rtrim, 1, 2, 0, trimFunc, AFFINITY_TEXT),
- FUNCTION(rtrim, 2, 2, 0, trimFunc, AFFINITY_TEXT),
- FUNCTION(trim, 1, 3, 0, trimFunc, AFFINITY_TEXT),
- FUNCTION(trim, 2, 3, 0, trimFunc, AFFINITY_TEXT),
+ FIELD_TYPE_INTEGER),
+ FUNCTION(ltrim, 1, 1, 0, trimFunc, FIELD_TYPE_STRING),
+ FUNCTION(ltrim, 2, 1, 0, trimFunc, FIELD_TYPE_STRING),
+ FUNCTION(rtrim, 1, 2, 0, trimFunc, FIELD_TYPE_STRING),
+ FUNCTION(rtrim, 2, 2, 0, trimFunc, FIELD_TYPE_STRING),
+ FUNCTION(trim, 1, 3, 0, trimFunc, FIELD_TYPE_STRING),
+ FUNCTION(trim, 2, 3, 0, trimFunc, FIELD_TYPE_STRING),
FUNCTION(min, -1, 0, 1, minmaxFunc, 0),
FUNCTION(min, 0, 0, 1, 0, 0),
AGGREGATE2(min, 1, 0, 1, minmaxStep, minMaxFinalize,
@@ -1724,56 +1724,49 @@ sqlite3RegisterBuiltinFunctions(void)
AGGREGATE2(max, 1, 1, 1, minmaxStep, minMaxFinalize,
SQLITE_FUNC_MINMAX, 0),
FUNCTION2(typeof, 1, 0, 0, typeofFunc, SQLITE_FUNC_TYPEOF,
- AFFINITY_TEXT),
+ FIELD_TYPE_STRING),
FUNCTION2(length, 1, 0, 0, lengthFunc, SQLITE_FUNC_LENGTH,
- AFFINITY_INTEGER),
- FUNCTION(instr, 2, 0, 0, instrFunc, AFFINITY_INTEGER),
- FUNCTION(printf, -1, 0, 0, printfFunc, AFFINITY_TEXT),
- FUNCTION(unicode, 1, 0, 0, unicodeFunc, AFFINITY_TEXT),
- FUNCTION(char, -1, 0, 0, charFunc, AFFINITY_TEXT),
- FUNCTION(abs, 1, 0, 0, absFunc, AFFINITY_REAL),
+ FIELD_TYPE_INTEGER),
+ FUNCTION(instr, 2, 0, 0, instrFunc, FIELD_TYPE_INTEGER),
+ FUNCTION(printf, -1, 0, 0, printfFunc, FIELD_TYPE_STRING),
+ FUNCTION(unicode, 1, 0, 0, unicodeFunc, FIELD_TYPE_STRING),
+ FUNCTION(char, -1, 0, 0, charFunc, FIELD_TYPE_STRING),
+ FUNCTION(abs, 1, 0, 0, absFunc, FIELD_TYPE_NUMBER),
#ifndef SQLITE_OMIT_FLOATING_POINT
- FUNCTION(round, 1, 0, 0, roundFunc, AFFINITY_INTEGER),
- FUNCTION(round, 2, 0, 0, roundFunc, AFFINITY_INTEGER),
+ FUNCTION(round, 1, 0, 0, roundFunc, FIELD_TYPE_INTEGER),
+ FUNCTION(round, 2, 0, 0, roundFunc, FIELD_TYPE_INTEGER),
#endif
- FUNCTION(upper, 1, 0, 1, UpperICUFunc, AFFINITY_TEXT),
- FUNCTION(lower, 1, 0, 1, LowerICUFunc, AFFINITY_TEXT),
- FUNCTION(hex, 1, 0, 0, hexFunc, AFFINITY_TEXT),
+ FUNCTION(upper, 1, 0, 1, UpperICUFunc, FIELD_TYPE_STRING),
+ FUNCTION(lower, 1, 0, 1, LowerICUFunc, FIELD_TYPE_STRING),
+ FUNCTION(hex, 1, 0, 0, hexFunc, FIELD_TYPE_STRING),
FUNCTION2(ifnull, 2, 0, 0, noopFunc, SQLITE_FUNC_COALESCE,
- AFFINITY_INTEGER),
- VFUNCTION(random, 0, 0, 0, randomFunc, AFFINITY_REAL),
- VFUNCTION(randomblob, 1, 0, 0, randomBlob, AFFINITY_BLOB),
+ FIELD_TYPE_INTEGER),
+ VFUNCTION(random, 0, 0, 0, randomFunc, FIELD_TYPE_NUMBER),
+ VFUNCTION(randomblob, 1, 0, 0, randomBlob, FIELD_TYPE_SCALAR),
FUNCTION(nullif, 2, 0, 1, nullifFunc, 0),
- FUNCTION(version, 0, 0, 0, sql_func_version, AFFINITY_TEXT),
- FUNCTION(quote, 1, 0, 0, quoteFunc, AFFINITY_TEXT),
- VFUNCTION(row_count, 0, 0, 0, sql_row_count, AFFINITY_INTEGER),
- FUNCTION(replace, 3, 0, 0, replaceFunc, AFFINITY_TEXT),
- FUNCTION(zeroblob, 1, 0, 0, zeroblobFunc, AFFINITY_BLOB),
- FUNCTION(substr, 2, 0, 0, substrFunc, AFFINITY_TEXT),
- FUNCTION(substr, 3, 0, 0, substrFunc, AFFINITY_TEXT),
+ FUNCTION(version, 0, 0, 0, sql_func_version, FIELD_TYPE_STRING),
+ FUNCTION(quote, 1, 0, 0, quoteFunc, FIELD_TYPE_STRING),
+ VFUNCTION(row_count, 0, 0, 0, sql_row_count, FIELD_TYPE_INTEGER),
+ FUNCTION(replace, 3, 0, 0, replaceFunc, FIELD_TYPE_STRING),
+ FUNCTION(zeroblob, 1, 0, 0, zeroblobFunc, FIELD_TYPE_SCALAR),
+ FUNCTION(substr, 2, 0, 0, substrFunc, FIELD_TYPE_STRING),
+ FUNCTION(substr, 3, 0, 0, substrFunc, FIELD_TYPE_STRING),
AGGREGATE(sum, 1, 0, 0, sumStep, sumFinalize, 0),
AGGREGATE(total, 1, 0, 0, sumStep, totalFinalize, 0),
AGGREGATE(avg, 1, 0, 0, sumStep, avgFinalize, 0),
AGGREGATE2(count, 0, 0, 0, countStep, countFinalize,
- SQLITE_FUNC_COUNT, AFFINITY_INTEGER),
+ SQLITE_FUNC_COUNT, FIELD_TYPE_INTEGER),
AGGREGATE(count, 1, 0, 0, countStep, countFinalize,
- AFFINITY_INTEGER),
+ FIELD_TYPE_INTEGER),
AGGREGATE(group_concat, 1, 0, 0, groupConcatStep,
- groupConcatFinalize, AFFINITY_TEXT),
+ groupConcatFinalize, FIELD_TYPE_STRING),
AGGREGATE(group_concat, 2, 0, 0, groupConcatStep,
- groupConcatFinalize, AFFINITY_TEXT),
-
-#ifdef SQLITE_CASE_SENSITIVE_LIKE
- LIKEFUNC(like, 2, 0,SQLITE_FUNC_LIKE | SQLITE_FUNC_CASE,
- AFFINITY_INTEGER),
- LIKEFUNC(like, 3, 0,SQLITE_FUNC_LIKE | SQLITE_FUNC_CASE,
- AFFINITY_INTEGER),
-#else
+ groupConcatFinalize, FIELD_TYPE_STRING),
+
LIKEFUNC(like, 2, 1, SQLITE_FUNC_LIKE,
- AFFINITY_INTEGER),
+ FIELD_TYPE_INTEGER),
LIKEFUNC(like, 3, 1, SQLITE_FUNC_LIKE,
- AFFINITY_INTEGER),
-#endif
+ FIELD_TYPE_INTEGER),
#ifdef SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
FUNCTION(unknown, -1, 0, 0, unknownFunc, 0),
#endif
diff --git a/src/box/sql/main.c b/src/box/sql/main.c
index 8574d6464..c009b9c63 100644
--- a/src/box/sql/main.c
+++ b/src/box/sql/main.c
@@ -750,7 +750,7 @@ sqlite3_interrupt(sqlite3 * db)
int
sqlite3CreateFunc(sqlite3 * db,
const char *zFunctionName,
- enum affinity_type type,
+ enum field_type type,
int nArg,
int flags,
void *pUserData,
@@ -820,7 +820,7 @@ sqlite3CreateFunc(sqlite3 * db,
int
sqlite3_create_function_v2(sqlite3 * db,
const char *zFunc,
- enum affinity_type type,
+ enum field_type type,
int nArg,
int flags,
void *p,
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index e2d630929..2a7223fff 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -732,7 +732,7 @@ sqlite3_memory_used(void);
int
sqlite3_create_function_v2(sqlite3 * db,
const char *zFunctionName,
- enum affinity_type type,
+ enum field_type type,
int nArg,
int flags,
void *pApp,
@@ -1660,7 +1660,7 @@ struct FuncDef {
FuncDestructor *pDestructor; /* Reference counted destructor function */
} u;
/* Return type. */
- enum affinity_type ret_type;
+ enum field_type ret_type;
};
/*
@@ -4569,7 +4569,7 @@ void sqlite3RegisterLikeFunctions(sqlite3 *, int);
int
sql_is_like_func(struct sqlite3 *db, struct Expr *expr, int *is_like_ci);
-int sqlite3CreateFunc(sqlite3 *, const char *, enum affinity_type,
+int sqlite3CreateFunc(sqlite3 *, const char *, enum field_type,
int, int, void *,
void (*)(sqlite3_context *, int, sqlite3_value **),
void (*)(sqlite3_context *, int, sqlite3_value **),
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (3 preceding siblings ...)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 4/8] sql: replace affinity with field type for func Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
2018-12-28 9:34 ` [tarantool-patches] [PATCH 6/8] sql: replace affinity with field type in struct Expr Nikita Pettik
` (4 subsequent siblings)
9 siblings, 1 reply; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
This stage of affinity removal requires introducing of auxiliary
intermediate function to convert array of affinity values to field type
values. The rest of job done in this commit is a straightforward
refactoring.
Part of #3698
---
src/box/sql/analyze.c | 5 +-
src/box/sql/build.c | 13 +++++
src/box/sql/delete.c | 16 +++----
src/box/sql/expr.c | 21 ++++----
src/box/sql/fkey.c | 3 +-
src/box/sql/insert.c | 31 ++++++++----
src/box/sql/select.c | 17 +++++--
src/box/sql/sqliteInt.h | 13 +++--
src/box/sql/update.c | 17 ++++---
src/box/sql/vdbe.c | 92 ++++++++++++++++--------------------
src/box/sql/vdbeInt.h | 2 +-
src/box/sql/vdbemem.c | 18 +++----
src/box/sql/where.c | 2 +-
src/box/sql/wherecode.c | 6 +--
test/sql-tap/cast.test.lua | 10 ++--
test/sql-tap/tkt-80e031a00f.test.lua | 8 ++--
16 files changed, 156 insertions(+), 118 deletions(-)
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index 51c63fa7a..a04dc8681 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -993,9 +993,10 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space)
sqlite3VdbeAddOp2(v, OP_Next, idx_cursor, next_row_addr);
/* Add the entry to the stat1 table. */
callStatGet(v, stat4_reg, STAT_GET_STAT1, stat1_reg);
- assert("BBB"[0] == AFFINITY_TEXT);
+ char types[3] = { FIELD_TYPE_STRING, FIELD_TYPE_STRING,
+ FIELD_TYPE_STRING };
sqlite3VdbeAddOp4(v, OP_MakeRecord, tab_name_reg, 3, tmp_reg,
- "BBB", 0);
+ types, 3);
sqlite3VdbeAddOp4(v, OP_IdxInsert, tmp_reg, 0, 0,
(char *)stat1, P4_SPACEPTR);
/* Add the entries to the stat4 table. */
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index e51e2db2a..b3f98c317 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -520,6 +520,19 @@ sql_field_type_to_affinity(enum field_type field_type)
}
}
+char *
+sql_affinity_str_to_field_type_str(const char *affinity_str)
+{
+ if (affinity_str == NULL)
+ return NULL;
+ size_t len = strlen(affinity_str) + 1;
+ char *type_str = (char *) sqlite3DbMallocRaw(sql_get(), len);
+ for (uint32_t i = 0; i < len - 1; ++i)
+ type_str[i] = sql_affinity_to_field_type(affinity_str[i]);
+ type_str[len - 1] = '\0';
+ return type_str;
+}
+
/*
* Add a new column to the table currently being constructed.
*
diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
index f9c42fdec..7b0d6b2fd 100644
--- a/src/box/sql/delete.c
+++ b/src/box/sql/delete.c
@@ -332,12 +332,12 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
*/
key_len = 0;
struct index *pk = space_index(space, 0);
- const char *zAff = is_view ? NULL :
- sql_space_index_affinity_str(parse->db,
- space->def,
- pk->def);
+ char *type_str = is_view ? NULL :
+ sql_index_type_str(parse->db,
+ pk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, reg_pk, pk_len,
- reg_key, zAff, pk_len);
+ reg_key, type_str, is_view ? 0 :
+ P4_DYNAMIC);
/* Set flag to save memory allocating one
* by malloc.
*/
@@ -592,13 +592,13 @@ sql_generate_index_key(struct Parse *parse, struct index *index, int cursor,
* is an integer, then it might be stored in the
* table as an integer (using a compact
* representation) then converted to REAL by an
- * OP_RealAffinity opcode. But we are getting
+ * OP_Realify opcode. But we are getting
* ready to store this value back into an index,
* where it should be converted by to INTEGER
- * again. So omit the OP_RealAffinity opcode if
+ * again. So omit the OP_Realify opcode if
* it is present
*/
- sqlite3VdbeDeletePriorOpcode(v, OP_RealAffinity);
+ sqlite3VdbeDeletePriorOpcode(v, OP_Realify);
}
if (reg_out != 0)
sqlite3VdbeAddOp3(v, OP_MakeRecord, reg_base, col_cnt, reg_out);
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 917e6e30b..c823c5a06 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -2146,10 +2146,10 @@ sqlite3ExprCanBeNull(const Expr * p)
/*
* Return TRUE if the given expression is a constant which would be
- * unchanged by OP_Affinity with the affinity given in the second
+ * unchanged by OP_ApplyType with the type given in the second
* argument.
*
- * This routine is used to determine if the OP_Affinity operation
+ * This routine is used to determine if the OP_ApplyType operation
* can be omitted. When in doubt return FALSE. A false negative
* is harmless. A false positive, however, can result in the wrong
* answer.
@@ -2858,8 +2858,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
jmpIfDynamic = -1;
}
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
+ char type =
+ sql_affinity_to_field_type(affinity);
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
- 1, r2, &affinity, 1);
+ 1, r2, &type,
+ 1);
sqlite3ExprCacheAffinityChange(pParse,
r3, 1);
sqlite3VdbeAddOp2(v, OP_IdxInsert, r2,
@@ -3172,7 +3175,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
+ char *type_str = sql_affinity_str_to_field_type_str(zAff);
+ sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, type_str, nVector);
if (destIfFalse == destIfNull) {
/* Combine Step 3 and Step 5 into a single opcode */
sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,
@@ -3700,7 +3704,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
pCol->iSorterColumn, target);
if (pCol->space_def->fields[pExpr->iAgg].type ==
FIELD_TYPE_NUMBER)
- sqlite3VdbeAddOp1(v, OP_RealAffinity,
+ sqlite3VdbeAddOp1(v, OP_Realify,
target);
return target;
}
@@ -3799,7 +3803,8 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
sqlite3VdbeAddOp2(v, OP_SCopy, inReg, target);
inReg = target;
}
- sqlite3VdbeAddOp2(v, OP_Cast, target, pExpr->affinity);
+ sqlite3VdbeAddOp2(v, OP_Cast, target,
+ sql_affinity_to_field_type(pExpr->affinity));
testcase(usedAsColumnCache(pParse, inReg, inReg));
sqlite3ExprCacheAffinityChange(pParse, inReg, 1);
return inReg;
@@ -4235,14 +4240,14 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
#ifndef SQLITE_OMIT_FLOATING_POINT
/* If the column has REAL affinity, it may currently be stored as an
- * integer. Use OP_RealAffinity to make sure it is really real.
+ * integer. Use OP_Realify to make sure it is really real.
*
* EVIDENCE-OF: R-60985-57662 SQLite will convert the value back to
* floating point when extracting it from the record.
*/
if (pExpr->iColumn >= 0 && def->fields[
pExpr->iColumn].affinity == AFFINITY_REAL) {
- sqlite3VdbeAddOp1(v, OP_RealAffinity, target);
+ sqlite3VdbeAddOp1(v, OP_Realify, target);
}
#endif
break;
diff --git a/src/box/sql/fkey.c b/src/box/sql/fkey.c
index 4e3270f0c..a6a8a24dd 100644
--- a/src/box/sql/fkey.c
+++ b/src/box/sql/fkey.c
@@ -256,8 +256,7 @@ fkey_lookup_parent(struct Parse *parse_context, struct space *parent,
struct index *idx = space_index(parent, referenced_idx);
assert(idx != NULL);
sqlite3VdbeAddOp4(v, OP_MakeRecord, temp_regs, field_count, rec_reg,
- sql_space_index_affinity_str(parse_context->db,
- parent->def, idx->def),
+ sql_index_type_str(parse_context->db, idx->def),
P4_DYNAMIC);
sqlite3VdbeAddOp4Int(v, OP_Found, cursor, ok_label, rec_reg, 0);
sqlite3ReleaseTempReg(parse_context, rec_reg);
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index 6b76bb6da..fd36e2786 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -68,17 +68,30 @@ sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
return aff;
}
+char *
+sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def)
+{
+ uint32_t column_count = idx_def->key_def->part_count;
+ char *types = (char *) sqlite3DbMallocRaw(db, column_count + 1);
+ if (types == NULL)
+ return NULL;
+ for (uint32_t i = 0; i < column_count; i++)
+ types[i] = idx_def->key_def->parts[i].type;
+ types[column_count] = '\0';
+ return types;
+}
+
void
-sql_emit_table_affinity(struct Vdbe *v, struct space_def *def, int reg)
+sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg)
{
assert(reg > 0);
struct sqlite3 *db = sqlite3VdbeDb(v);
uint32_t field_count = def->field_count;
- char *colls_aff = (char *) sqlite3DbMallocZero(db, field_count + 1);
- if (colls_aff == NULL)
+ char *colls_type = (char *) sqlite3DbMallocZero(db, field_count + 1);
+ if (colls_type == NULL)
return;
for (uint32_t i = 0; i < field_count; ++i) {
- colls_aff[i] = def->fields[i].affinity;
+ colls_type[i] = def->fields[i].type;
/*
* Force INTEGER type to handle queries like:
* CREATE TABLE t1 (id INT PRIMARY KEY);
@@ -86,12 +99,12 @@ sql_emit_table_affinity(struct Vdbe *v, struct space_def *def, int reg)
*
* In this case 1.123 should be truncated to 1.
*/
- if (colls_aff[i] == AFFINITY_INTEGER) {
+ if (colls_type[i] == FIELD_TYPE_INTEGER) {
sqlite3VdbeAddOp2(v, OP_Cast, reg + i,
- AFFINITY_INTEGER);
+ FIELD_TYPE_INTEGER);
}
}
- sqlite3VdbeAddOp4(v, OP_Affinity, reg, field_count, 0, colls_aff,
+ sqlite3VdbeAddOp4(v, OP_ApplyType, reg, field_count, 0, colls_type,
P4_DYNAMIC);
}
@@ -616,7 +629,7 @@ sqlite3Insert(Parse * pParse, /* Parser context */
* table column affinities.
*/
if (!is_view)
- sql_emit_table_affinity(v, pTab->def, regCols + 1);
+ sql_emit_table_types(v, pTab->def, regCols + 1);
/* Fire BEFORE or INSTEAD OF triggers */
vdbe_code_row_trigger(pParse, trigger, TK_INSERT, 0,
@@ -964,7 +977,7 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct Table *tab,
sqlite3VdbeResolveLabel(v, all_ok);
}
}
- sql_emit_table_affinity(v, tab->def, new_tuple_reg);
+ sql_emit_table_types(v, tab->def, new_tuple_reg);
/*
* If PK is marked as INTEGER, use it as strict type,
* not as affinity. Emit code for type checking.
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 40336e679..0ee40093f 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1202,9 +1202,12 @@ selectInnerLoop(Parse * pParse, /* The parser context */
int r1 = sqlite3GetTempReg(pParse);
assert(sqlite3Strlen30(pDest->zAffSdst) ==
(unsigned int)nResultCol);
+ char *type_str =
+ sql_affinity_str_to_field_type_str(
+ pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
- nResultCol, r1,
- pDest->zAffSdst, nResultCol);
+ nResultCol, r1, type_str,
+ P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(pParse,
regResult,
nResultCol);
@@ -1626,8 +1629,11 @@ generateSortTail(Parse * pParse, /* Parsing context */
case SRT_Set:{
assert((unsigned int)nColumn ==
sqlite3Strlen30(pDest->zAffSdst));
+
+ const char *type_str =
+ sql_affinity_str_to_field_type_str(pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
- regTupleid, pDest->zAffSdst, nColumn);
+ regTupleid, type_str, P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(pParse, regRow, nColumn);
sqlite3VdbeAddOp2(v, OP_IdxInsert, regTupleid, pDest->reg_eph);
break;
@@ -3062,9 +3068,10 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
int r1;
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
+ const char *type_str =
+ sql_affinity_str_to_field_type_str(dest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
- in->nSdst, r1, dest->zAffSdst,
- in->nSdst);
+ in->nSdst, r1, type_str, P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(parse, in->iSdst,
in->nSdst);
sqlite3VdbeAddOp2(v, OP_IdxInsert, r1, dest->reg_eph);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 2a7223fff..690fa6431 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3439,6 +3439,9 @@ sql_affinity_to_field_type(enum affinity_type affinity);
enum affinity_type
sql_field_type_to_affinity(enum field_type field_type);
+char *
+sql_affinity_str_to_field_type_str(const char *affinity_str);
+
/**
* Compile view, i.e. create struct Select from
* 'CREATE VIEW...' string, and assign cursors to each table from
@@ -4235,16 +4238,20 @@ char *
sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
struct index_def *idx_def);
+/** Return string consisting of fields types of given index. */
+char *
+sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
+
/**
- * Code an OP_Affinity opcode that will set affinities
+ * Code an OP_ApplyType opcode that will force types
* for given range of register starting from @reg.
*
* @param v VDBE.
* @param def Definition of table to be used.
- * @param reg Register where affinities will be placed.
+ * @param reg Register where types will be placed.
*/
void
-sql_emit_table_affinity(struct Vdbe *v, struct space_def *def, int reg);
+sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
/**
* Return superposition of two affinities.
diff --git a/src/box/sql/update.c b/src/box/sql/update.c
index 0e2d0fde8..41eae1550 100644
--- a/src/box/sql/update.c
+++ b/src/box/sql/update.c
@@ -44,7 +44,7 @@ sqlite3ColumnDefault(Vdbe *v, struct space_def *def, int i, int ireg)
assert(def != 0);
if (!def->opts.is_view) {
sqlite3_value *pValue = 0;
- char affinity = def->fields[i].affinity;
+ enum field_type type = def->fields[i].type;
VdbeComment((v, "%s.%s", def->name, def->fields[i].name));
assert(i < (int)def->field_count);
@@ -52,14 +52,14 @@ sqlite3ColumnDefault(Vdbe *v, struct space_def *def, int i, int ireg)
assert(def->fields != NULL && i < (int)def->field_count);
if (def->fields != NULL)
expr = def->fields[i].default_value_expr;
- sqlite3ValueFromExpr(sqlite3VdbeDb(v), expr, affinity,
+ sqlite3ValueFromExpr(sqlite3VdbeDb(v), expr, type,
&pValue);
if (pValue) {
sqlite3VdbeAppendP4(v, pValue, P4_MEM);
}
#ifndef SQLITE_OMIT_FLOATING_POINT
- if (affinity == AFFINITY_REAL) {
- sqlite3VdbeAddOp1(v, OP_RealAffinity, ireg);
+ if (type == FIELD_TYPE_NUMBER) {
+ sqlite3VdbeAddOp1(v, OP_Realify, ireg);
}
#endif
}
@@ -274,11 +274,10 @@ sqlite3Update(Parse * pParse, /* The parser context */
nKey = pk_part_count;
regKey = iPk;
} else {
- const char *zAff = is_view ? 0 :
- sql_space_index_affinity_str(pParse->db, def,
- pPk->def);
+ char *type_str = is_view ? NULL :
+ sql_index_type_str(pParse->db, pPk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, pk_part_count,
- regKey, zAff, pk_part_count);
+ regKey, type_str, is_view ? 0 : P4_DYNAMIC);
/*
* Set flag to save memory allocating one by
* malloc.
@@ -390,7 +389,7 @@ sqlite3Update(Parse * pParse, /* The parser context */
* verified. One could argue that this is wrong.
*/
if (tmask & TRIGGER_BEFORE) {
- sql_emit_table_affinity(v, pTab->def, regNew);
+ sql_emit_table_types(v, pTab->def, regNew);
vdbe_code_row_trigger(pParse, trigger, TK_UPDATE, pChanges,
TRIGGER_BEFORE, pTab, regOldPk,
on_error, labelContinue);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4345af24e..369fb4b79 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -306,32 +306,35 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
}
/**
- * Processing is determine by the affinity parameter:
+ * Processing is determine by the field type parameter:
*
- * AFFINITY_INTEGER:
- * AFFINITY_REAL:
- * Try to convert mem to an integer representation or a
- * floating-point representation if an integer representation
- * is not possible. Note that the integer representation is
- * always preferred, even if the affinity is REAL, because
- * an integer representation is more space efficient on disk.
+ * INTEGER:
+ * If memory holds floating point value and it can be
+ * converted without loss (2.0 - > 2), it's type is
+ * changed to INT. Otherwise, simply return success status.
*
- * AFFINITY_TEXT:
- * Convert mem to a text representation.
+ * NUMBER:
+ * If memory holds INT or floating point value,
+ * no actions take place.
*
- * AFFINITY_BLOB:
- * No-op. mem is unchanged.
+ * STRING:
+ * Convert mem to a string representation.
*
- * @param record The value to apply affinity to.
- * @param affinity The affinity to be applied.
+ * SCALAR:
+ * Mem is unchanged, but flat is set to BLOB.
+ *
+ * @param record The value to apply type to.
+ * @param type_t The type to be applied.
*/
static int
-mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
+mem_apply_type(struct Mem *record, enum field_type f_type)
{
if ((record->flags & MEM_Null) != 0)
return 0;
- switch (affinity) {
- case AFFINITY_INTEGER:
+ assert(f_type < field_type_MAX);
+ switch (f_type) {
+ case FIELD_TYPE_INTEGER:
+ case FIELD_TYPE_UNSIGNED:
if ((record->flags & MEM_Int) == MEM_Int)
return 0;
if ((record->flags & MEM_Real) == MEM_Real) {
@@ -343,11 +346,11 @@ mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
return 0;
}
return sqlite3VdbeMemIntegerify(record, false);
- case AFFINITY_REAL:
+ case FIELD_TYPE_NUMBER:
if ((record->flags & (MEM_Real | MEM_Int)) != 0)
return 0;
return sqlite3VdbeMemRealify(record);
- case AFFINITY_TEXT:
+ case FIELD_TYPE_STRING:
/*
* Only attempt the conversion to TEXT if there is
* an integer or real representation (BLOB and
@@ -359,7 +362,7 @@ mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
}
record->flags &= ~(MEM_Real | MEM_Int);
return 0;
- case AFFINITY_BLOB:
+ case FIELD_TYPE_SCALAR:
if (record->flags & (MEM_Str | MEM_Blob))
record->flags |= MEM_Blob;
return 0;
@@ -385,7 +388,7 @@ int sqlite3_value_numeric_type(sqlite3_value *pVal) {
}
/*
- * Exported version of mem_apply_affinity(). This one works on sqlite3_value*,
+ * Exported version of mem_apply_type(). This one works on sqlite3_value*,
* not the internal Mem* type.
*/
void
@@ -393,7 +396,7 @@ sqlite3ValueApplyAffinity(
sqlite3_value *pVal,
u8 affinity)
{
- mem_apply_affinity((Mem *) pVal, affinity);
+ mem_apply_type((Mem *) pVal, affinity);
}
/*
@@ -1946,7 +1949,7 @@ case OP_AddImm: { /* in1 */
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
if ((pIn1->flags & MEM_Int)==0) {
- mem_apply_affinity(pIn1, AFFINITY_INTEGER);
+ mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
VdbeBranchTaken((pIn1->flags&MEM_Int)==0, 2);
if ((pIn1->flags & MEM_Int)==0) {
if (pOp->p2==0) {
@@ -1962,16 +1965,16 @@ case OP_MustBeInt: { /* jump, in1 */
}
#ifndef SQLITE_OMIT_FLOATING_POINT
-/* Opcode: RealAffinity P1 * * * *
+/* Opcode: Realify P1 * * * *
*
* If register P1 holds an integer convert it to a real value.
*
* This opcode is used when extracting information from a column that
- * has REAL affinity. Such column values may still be stored as
+ * has float type. Such column values may still be stored as
* integers, for space efficiency, but after extraction we want them
* to have only a real value.
*/
-case OP_RealAffinity: { /* in1 */
+case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
if (pIn1->flags & MEM_Int) {
sqlite3VdbeMemRealify(pIn1);
@@ -1982,7 +1985,7 @@ case OP_RealAffinity: { /* in1 */
#ifndef SQLITE_OMIT_CAST
/* Opcode: Cast P1 P2 * * *
- * Synopsis: affinity(r[P1])
+ * Synopsis: type(r[P1])
*
* Force the value in register P1 to be the type defined by P2.
*
@@ -1997,11 +2000,6 @@ case OP_RealAffinity: { /* in1 */
* A NULL value is not changed by this routine. It remains NULL.
*/
case OP_Cast: { /* in1 */
- assert(pOp->p2>=AFFINITY_BLOB && pOp->p2<=AFFINITY_REAL);
- testcase( pOp->p2==AFFINITY_TEXT);
- testcase( pOp->p2==AFFINITY_BLOB);
- testcase( pOp->p2==AFFINITY_INTEGER);
- testcase( pOp->p2==AFFINITY_REAL);
pIn1 = &aMem[pOp->p1];
rc = ExpandBlob(pIn1);
if (rc != 0)
@@ -2011,7 +2009,7 @@ case OP_Cast: { /* in1 */
if (rc == 0)
break;
diag_set(ClientError, ER_SQL_TYPE_MISMATCH, sqlite3_value_text(pIn1),
- affinity_type_str(pOp->p2));
+ field_type_strs[pOp->p2]);
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
@@ -2225,7 +2223,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
default: res2 = res>=0; break;
}
- /* Undo any changes made by mem_apply_affinity() to the input registers. */
+ /* Undo any changes made by mem_apply_type() to the input registers. */
assert((pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn));
pIn1->flags = flags1;
assert((pIn3->flags & MEM_Dyn) == (flags3 & MEM_Dyn));
@@ -2814,21 +2812,19 @@ case OP_Column: {
* string indicates the column affinity that should be used for the nth
* memory cell in the range.
*/
-case OP_Affinity: {
- const char *zAffinity; /* The affinity to be applied */
- char cAff; /* A single character of affinity */
-
- zAffinity = pOp->p4.z;
- assert(zAffinity!=0);
- assert(zAffinity[pOp->p2]==0);
+case OP_ApplyType: {
+ const char *type_str = pOp->p4.z;
+ assert(type_str != NULL);
+ assert(type_str[pOp->p2] == '\0');
pIn1 = &aMem[pOp->p1];
- while( (cAff = *(zAffinity++))!=0) {
+ char type;
+ while((type = *(type_str++)) != '\0') {
assert(pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)]);
assert(memIsValid(pIn1));
- if (mem_apply_affinity(pIn1, cAff) != 0) {
+ if (mem_apply_type(pIn1, type) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sqlite3_value_text(pIn1),
- affinity_type_str(cAff));
+ field_type_strs[type]);
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
@@ -2848,10 +2844,7 @@ case OP_Affinity: {
* string indicates the column affinity that should be used for the nth
* field of the index key.
*
- * The mapping from character to affinity is given by the AFFINITY_
- * macros defined in sqliteInt.h.
- *
- * If P4 is NULL then all index fields have the affinity BLOB.
+ * If P4 is NULL then all index fields have type SCALAR.
*
* If P5 is not NULL then record under construction is intended to be inserted
* into ephemeral space. Thus, sort of memory optimization can be performed.
@@ -2899,8 +2892,7 @@ case OP_MakeRecord: {
if (zAffinity) {
pRec = pData0;
do{
- mem_apply_affinity(pRec++, *(zAffinity++));
- assert(zAffinity[0]==0 || pRec<=pLast);
+ mem_apply_type(pRec++, *(zAffinity++));
}while( zAffinity[0]);
}
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 50bc35b2b..879ba34d0 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -483,7 +483,7 @@ int sqlite3VdbeRealValue(Mem *, double *);
int sqlite3VdbeIntegerAffinity(Mem *);
int sqlite3VdbeMemRealify(Mem *);
int sqlite3VdbeMemNumerify(Mem *);
-int sqlite3VdbeMemCast(Mem *, u8);
+int sqlite3VdbeMemCast(Mem *, enum field_type type);
int sqlite3VdbeMemFromBtree(BtCursor *, u32, u32, Mem *);
void sqlite3VdbeMemRelease(Mem * p);
int sqlite3VdbeMemFinalize(Mem *, FuncDef *);
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index bb4d91aed..cd71641b0 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -598,11 +598,12 @@ sqlite3VdbeMemNumerify(Mem * pMem)
* used (for example) to implement the SQL "cast()" operator.
*/
int
-sqlite3VdbeMemCast(Mem * pMem, u8 aff)
+sqlite3VdbeMemCast(Mem * pMem, enum field_type type)
{
+ assert(type < field_type_MAX);
if (pMem->flags & MEM_Null)
return SQLITE_OK;
- if ((pMem->flags & MEM_Blob) != 0 && aff == AFFINITY_REAL) {
+ if ((pMem->flags & MEM_Blob) != 0 && type == FIELD_TYPE_NUMBER) {
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Real);
pMem->u.r = pMem->u.i;
@@ -610,8 +611,8 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
}
return ! sqlite3AtoF(pMem->z, &pMem->u.r, pMem->n);
}
- switch (aff) {
- case AFFINITY_BLOB:
+ switch (type) {
+ case FIELD_TYPE_SCALAR:
if (pMem->flags & MEM_Blob)
return SQLITE_OK;
if (pMem->flags & MEM_Str) {
@@ -625,7 +626,7 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
return 0;
}
return SQLITE_ERROR;
- case AFFINITY_INTEGER:
+ case FIELD_TYPE_INTEGER:
if ((pMem->flags & MEM_Blob) != 0) {
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i,
pMem->n) != 0)
@@ -634,13 +635,13 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
return 0;
}
return sqlite3VdbeMemIntegerify(pMem, true);
- case AFFINITY_REAL:
+ case FIELD_TYPE_NUMBER:
return sqlite3VdbeMemRealify(pMem);
default:
- assert(aff == AFFINITY_TEXT);
+ assert(type == FIELD_TYPE_STRING);
assert(MEM_Str == (MEM_Blob >> 3));
pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
- sqlite3ValueApplyAffinity(pMem, AFFINITY_TEXT);
+ sqlite3ValueApplyAffinity(pMem, FIELD_TYPE_STRING);
assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
pMem->flags &= ~(MEM_Int | MEM_Real | MEM_Blob | MEM_Zero);
return SQLITE_OK;
@@ -1579,6 +1580,7 @@ sqlite3Stat4ProbeSetValue(Parse * pParse, /* Parse context */
u8 aff = sql_space_index_part_affinity(space->def, idx,
iVal + i);
alloc.iVal = iVal + i;
+ aff = sql_affinity_to_field_type(aff);
rc = stat4ValueFromExpr(pParse, pElem, aff, &alloc,
&pVal);
if (!pVal)
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 571b5af78..539296079 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
int nLower = -1;
int nUpper = index->def->opts.stat->sample_count + 1;
int rc = SQLITE_OK;
- u8 aff = sql_space_index_part_affinity(space->def, p, nEq);
+ u8 aff = p->key_def->parts[nEq].type;
sqlite3_value *p1 = 0; /* Value extracted from pLower */
sqlite3_value *p2 = 0; /* Value extracted from pUpper */
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index b124a1d53..aae5d6617 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -364,7 +364,7 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
}
/*
- * Code an OP_Affinity opcode to apply the column affinity string zAff
+ * Code an OP_ApplyType opcode to apply the column type string types
* to the n registers starting at base.
*
* As an optimization, AFFINITY_BLOB entries (which are no-ops) at the
@@ -396,9 +396,9 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
n--;
}
- /* Code the OP_Affinity opcode if there is anything left to do. */
if (n > 0) {
- sqlite3VdbeAddOp4(v, OP_Affinity, base, n, 0, zAff, n);
+ const char *type_str = sql_affinity_str_to_field_type_str(zAff);
+ sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, type_str, n);
sqlite3ExprCacheAffinityChange(pParse, base, n);
}
}
diff --git a/test/sql-tap/cast.test.lua b/test/sql-tap/cast.test.lua
index 51d557937..189ca8933 100755
--- a/test/sql-tap/cast.test.lua
+++ b/test/sql-tap/cast.test.lua
@@ -70,7 +70,7 @@ test:do_catchsql_test(
SELECT CAST(x'616263' AS numeric)
]], {
-- <cast-1.5>
- 1, 'Type mismatch: can not convert abc to real'
+ 1, 'Type mismatch: can not convert abc to number'
-- </cast-1.5>
})
@@ -450,7 +450,7 @@ test:do_catchsql_test(
SELECT CAST('123abc' AS numeric)
]], {
-- <cast-1.45>
- 1, 'Type mismatch: can not convert 123abc to real'
+ 1, 'Type mismatch: can not convert 123abc to number'
-- </cast-1.45>
})
@@ -480,7 +480,7 @@ test:do_catchsql_test(
SELECT CAST('123.5abc' AS numeric)
]], {
-- <cast-1.51>
- 1, 'Type mismatch: can not convert 123.5abc to real'
+ 1, 'Type mismatch: can not convert 123.5abc to number'
-- </cast-1.51>
})
@@ -561,7 +561,7 @@ test:do_catchsql_test(
SELECT CAST('abc' AS REAL)
]], {
-- <case-1.66>
- 1, 'Type mismatch: can not convert abc to real'
+ 1, 'Type mismatch: can not convert abc to number'
-- </case-1.66>
})
@@ -875,7 +875,7 @@ test:do_test(
]]
end, {
-- <cast-4.4>
- 1, 'Type mismatch: can not convert abc to real'
+ 1, 'Type mismatch: can not convert abc to number'
-- </cast-4.4>
})
diff --git a/test/sql-tap/tkt-80e031a00f.test.lua b/test/sql-tap/tkt-80e031a00f.test.lua
index 2d4f81798..8517a581f 100755
--- a/test/sql-tap/tkt-80e031a00f.test.lua
+++ b/test/sql-tap/tkt-80e031a00f.test.lua
@@ -346,7 +346,7 @@ test:do_catchsql_test(
SELECT 'hello' IN t1
]], {
-- <tkt-80e031a00f.27>
- 1, 'Type mismatch: can not convert hello to real'
+ 1, 'Type mismatch: can not convert hello to number'
-- </tkt-80e031a00f.27>
})
@@ -356,7 +356,7 @@ test:do_catchsql_test(
SELECT 'hello' NOT IN t1
]], {
-- <tkt-80e031a00f.28>
- 1, 'Type mismatch: can not convert hello to real'
+ 1, 'Type mismatch: can not convert hello to number'
-- </tkt-80e031a00f.28>
})
@@ -386,7 +386,7 @@ test:do_catchsql_test(
SELECT x'303132' IN t1
]], {
-- <tkt-80e031a00f.31>
- 1, 'Type mismatch: can not convert 012 to real'
+ 1, 'Type mismatch: can not convert 012 to number'
-- </tkt-80e031a00f.31>
})
@@ -396,7 +396,7 @@ test:do_catchsql_test(
SELECT x'303132' NOT IN t1
]], {
-- <tkt-80e031a00f.32>
- 1, 'Type mismatch: can not convert 012 to real'
+ 1, 'Type mismatch: can not convert 012 to number'
-- </tkt-80e031a00f.32>
})
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2018-12-28 9:34 ` [tarantool-patches] [PATCH 5/8] sql: replace field type with affinity for VDBE runtime Nikita Pettik
@ 2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-16 14:26 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2018-12-29 17:42 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 8 comments below.
> sql: replace field type with affinity for VDBE runtime
1. Maybe vice versa? Replace affinity with field type?
On 28/12/2018 12:34, Nikita Pettik wrote:
> This stage of affinity removal requires introducing of auxiliary
> intermediate function to convert array of affinity values to field type
> values. The rest of job done in this commit is a straightforward
> refactoring.
>
> Part of #3698
> ---
> src/box/sql/analyze.c | 5 +-
> src/box/sql/build.c | 13 +++++
> src/box/sql/delete.c | 16 +++----
> src/box/sql/expr.c | 21 ++++----
> src/box/sql/fkey.c | 3 +-
> src/box/sql/insert.c | 31 ++++++++----
> src/box/sql/select.c | 17 +++++--
> src/box/sql/sqliteInt.h | 13 +++--
> src/box/sql/update.c | 17 ++++---
> src/box/sql/vdbe.c | 92 ++++++++++++++++--------------------
> src/box/sql/vdbeInt.h | 2 +-
> src/box/sql/vdbemem.c | 18 +++----
> src/box/sql/where.c | 2 +-
> src/box/sql/wherecode.c | 6 +--
> test/sql-tap/cast.test.lua | 10 ++--
> test/sql-tap/tkt-80e031a00f.test.lua | 8 ++--
> 16 files changed, 156 insertions(+), 118 deletions(-)
>
> diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
> index 51c63fa7a..a04dc8681 100644
> --- a/src/box/sql/analyze.c
> +++ b/src/box/sql/analyze.c
> @@ -993,9 +993,10 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space)
> sqlite3VdbeAddOp2(v, OP_Next, idx_cursor, next_row_addr);
> /* Add the entry to the stat1 table. */
> callStatGet(v, stat4_reg, STAT_GET_STAT1, stat1_reg);
> - assert("BBB"[0] == AFFINITY_TEXT);
> + char types[3] = { FIELD_TYPE_STRING, FIELD_TYPE_STRING,
> + FIELD_TYPE_STRING };
2. How about explicit type? Not char[], but enum field_type[] ? It will
look much more readable and convenient, IMO. Here and in other places.
'type_str', used now in all places instead of affinity str, looks crutchy.
> sqlite3VdbeAddOp4(v, OP_MakeRecord, tab_name_reg, 3, tmp_reg,
> - "BBB", 0);
> + types, 3);
> sqlite3VdbeAddOp4(v, OP_IdxInsert, tmp_reg, 0, 0,
> (char *)stat1, P4_SPACEPTR);
> /* Add the entries to the stat4 table. */
> diff --git a/src/box/sql/build.c b/src/box/sql/build.c
> index e51e2db2a..b3f98c317 100644
> --- a/src/box/sql/build.c
> +++ b/src/box/sql/build.c
> @@ -520,6 +520,19 @@ sql_field_type_to_affinity(enum field_type field_type)
> }
> }
>
> +char *
2. So not char *, but enum field_type *. Ok?
> +sql_affinity_str_to_field_type_str(const char *affinity_str)
> +{
> + if (affinity_str == NULL)
> + return NULL;
> + size_t len = strlen(affinity_str) + 1;
> + char *type_str = (char *) sqlite3DbMallocRaw(sql_get(), len);
> + for (uint32_t i = 0; i < len - 1; ++i)
> + type_str[i] = sql_affinity_to_field_type(affinity_str[i]);
> + type_str[len - 1] = '\0';
2. Instead of 0 you can use field_type_MAX terminator, if we will move
to enum field_type[].
> + return type_str;
> +}
> +
> /*
> * Add a new column to the table currently being constructed.
> *
> diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
> index f9c42fdec..7b0d6b2fd 100644
> --- a/src/box/sql/delete.c
> +++ b/src/box/sql/delete.c
> @@ -332,12 +332,12 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
> */
> key_len = 0;
> struct index *pk = space_index(space, 0);
> - const char *zAff = is_view ? NULL :
> - sql_space_index_affinity_str(parse->db,
> - space->def,
> - pk->def);
> + char *type_str = is_view ? NULL :
> + sql_index_type_str(parse->db,
> + pk->def);
> sqlite3VdbeAddOp4(v, OP_MakeRecord, reg_pk, pk_len,
> - reg_key, zAff, pk_len);
> + reg_key, type_str, is_view ? 0 :
> + P4_DYNAMIC);
3. How did it work before your patch? Looks like it was a leak. Before the
patch, pk_len was passed instead of STATIC/DYNAMIC.
> /* Set flag to save memory allocating one
> * by malloc.
> */
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index 917e6e30b..c823c5a06 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -2858,8 +2858,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
> jmpIfDynamic = -1;
> }
> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
> + char type =
> + sql_affinity_to_field_type(affinity);
> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
> - 1, r2, &affinity, 1);
> + 1, r2, &type,
> + 1);
4. I do not understand. Is p4type of sqlite3VdbeAddOp4 of OP_MakeRecord
a length of an affinity string, or a type of its allocation? Or both?
Also, I wonder how does this code works:
if (zAffinity) {
pRec = pData0;
do{
mem_apply_type(pRec++, *(zAffinity++));
}while( zAffinity[0]);
}
in OP_MakeRecord. It assumes, that zAffinity is a null-terminated string,
but in the code above you pass one char, without zero-termination.
> sqlite3ExprCacheAffinityChange(pParse,
> r3, 1);
> sqlite3VdbeAddOp2(v, OP_IdxInsert, r2,
> @@ -3172,7 +3175,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
> * of the RHS using the LHS as a probe. If found, the result is
> * true.
> */
> - sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
> + char *type_str = sql_affinity_str_to_field_type_str(zAff);
> + sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, type_str, nVector);
5. type_str is dynamically allocated, nVector (== p4type) is > 0. But freeP4
function in vdbeaux.c does not know how to free p4type > 0. So it is definitely
a leak. Please, validate all places where dynamic type string is created.
> if (destIfFalse == destIfNull) {
> /* Combine Step 3 and Step 5 into a single opcode */
> sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 4345af24e..369fb4b79 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -306,32 +306,35 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
> }
>
> /**
> - * Processing is determine by the affinity parameter:
> + * Processing is determine by the field type parameter:
> *
> - * AFFINITY_INTEGER:
> - * AFFINITY_REAL:
> - * Try to convert mem to an integer representation or a
> - * floating-point representation if an integer representation
> - * is not possible. Note that the integer representation is
> - * always preferred, even if the affinity is REAL, because
> - * an integer representation is more space efficient on disk.
> + * INTEGER:
> + * If memory holds floating point value and it can be
> + * converted without loss (2.0 - > 2), it's type is
> + * changed to INT. Otherwise, simply return success status.
> *
> - * AFFINITY_TEXT:
> - * Convert mem to a text representation.
> + * NUMBER:
> + * If memory holds INT or floating point value,
> + * no actions take place.
> *
> - * AFFINITY_BLOB:
> - * No-op. mem is unchanged.
> + * STRING:
> + * Convert mem to a string representation.
> *
> - * @param record The value to apply affinity to.
> - * @param affinity The affinity to be applied.
> + * SCALAR:
> + * Mem is unchanged, but flat is set to BLOB.
6. flat -> flag?
> + *
> + * @param record The value to apply type to.
> + * @param type_t The type to be applied.
> */
> static int
> -mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
> +mem_apply_type(struct Mem *record, enum field_type f_type)
> {
> if ((record->flags & MEM_Null) != 0)
> return 0;
> - switch (affinity) {
> - case AFFINITY_INTEGER:
> + assert(f_type < field_type_MAX);
7. Double white-space.
> + switch (f_type) {
> + case FIELD_TYPE_INTEGER:
> + case FIELD_TYPE_UNSIGNED:
> if ((record->flags & MEM_Int) == MEM_Int)
> return 0;
> if ((record->flags & MEM_Real) == MEM_Real) {
> diff --git a/src/box/sql/where.c b/src/box/sql/where.c
> index 571b5af78..539296079 100644
> --- a/src/box/sql/where.c
> +++ b/src/box/sql/where.c
> @@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
> int nLower = -1;
> int nUpper = index->def->opts.stat->sample_count + 1;
> int rc = SQLITE_OK;
> - u8 aff = sql_space_index_part_affinity(space->def, p, nEq);
> + u8 aff = p->key_def->parts[nEq].type;
8. Why? Below in this function aff is used as affinity, not type.
>
> sqlite3_value *p1 = 0; /* Value extracted from pLower */
> sqlite3_value *p2 = 0; /* Value extracted from pUpper */
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-01-16 14:26 ` n.pettik
2019-01-22 15:41 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-16 14:26 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
Alongside with fixes which you requested, I found that columns
in form of new.a or old.a (within trigger) have wrong calculated type.
Fix is simple:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index c0957de70..d3a8644ce 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -65,6 +65,7 @@ sql_expr_type(struct Expr *pExpr)
return pExpr->type;
case TK_AGG_COLUMN:
case TK_COLUMN:
+ case TK_TRIGGER:
assert(pExpr->iColumn >= 0);
return pExpr->space_def->fields[pExpr->iColumn].type;
case TK_SELECT_COLUMN:
> Thanks for the patch! See 8 comments below.
>
>> sql: replace field type with affinity for VDBE runtime
>
> 1. Maybe vice versa? Replace affinity with field type?
You are right, fixed:
sql: replace affinity with field type for VDBE runtime
>> diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
>> index 51c63fa7a..a04dc8681 100644
>> --- a/src/box/sql/analyze.c
>> +++ b/src/box/sql/analyze.c
>> @@ -993,9 +993,10 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space)
>> sqlite3VdbeAddOp2(v, OP_Next, idx_cursor, next_row_addr);
>> /* Add the entry to the stat1 table. */
>> callStatGet(v, stat4_reg, STAT_GET_STAT1, stat1_reg);
>> - assert("BBB"[0] == AFFINITY_TEXT);
>> + char types[3] = { FIELD_TYPE_STRING, FIELD_TYPE_STRING,
>> + FIELD_TYPE_STRING };
>
> 2. How about explicit type? Not char[], but enum field_type[] ? It will
> look much more readable and convenient, IMO. Here and in other places.
>
> 'type_str', used now in all places instead of affinity str, looks crutchy.
Ok, I don’t mind, but then we need to add explicit cast to char*:
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index a04dc8681..aa6b28e3b 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -993,10 +993,12 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space)
sqlite3VdbeAddOp2(v, OP_Next, idx_cursor, next_row_addr);
/* Add the entry to the stat1 table. */
callStatGet(v, stat4_reg, STAT_GET_STAT1, stat1_reg);
- char types[3] = { FIELD_TYPE_STRING, FIELD_TYPE_STRING,
- FIELD_TYPE_STRING };
- sqlite3VdbeAddOp4(v, OP_MakeRecord, tab_name_reg, 3, tmp_reg,
- types, 3);
+ enum field_type types[4] = { FIELD_TYPE_STRING,
+ FIELD_TYPE_STRING,
+ FIELD_TYPE_STRING,
+ field_type_MAX };
+ sqlite3VdbeAddOp4(v, OP_MakeRecord, tab_name_reg, 4, tmp_reg,
+ (char *)types, sizeof(types));
sqlite3VdbeAddOp4(v, OP_IdxInsert, tmp_reg, 0, 0,
(char *)stat1, P4_SPACEPTR);
Other usages:
diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
index 7b0d6b2fd..ca6c49373 100644
--- a/src/box/sql/delete.c
+++ b/src/box/sql/delete.c
@@ -332,12 +332,12 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
*/
key_len = 0;
struct index *pk = space_index(space, 0);
- char *type_str = is_view ? NULL :
- sql_index_type_str(parse->db,
- pk->def);
+ enum field_type *types = is_view ? NULL :
+ sql_index_type_str(parse->db,
+ pk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, reg_pk, pk_len,
- reg_key, type_str, is_view ? 0 :
- P4_DYNAMIC);
+ reg_key, (char *)types, is_view ? 0 :
+ P4_DYNAMIC);
diff --git a/src/box/sql/fkey.c b/src/box/sql/fkey.c
index a6a8a24dd..8aa8ea01e 100644
--- a/src/box/sql/fkey.c
+++ b/src/box/sql/fkey.c
@@ -256,8 +256,8 @@ fkey_lookup_parent(struct Parse *parse_context, struct space *parent,
struct index *idx = space_index(parent, referenced_idx);
assert(idx != NULL);
sqlite3VdbeAddOp4(v, OP_MakeRecord, temp_regs, field_count, rec_reg,
- sql_index_type_str(parse_context->db, idx->def),
- P4_DYNAMIC);
+ (char *) sql_index_type_str(parse_context->db,
+ idx->def), P4_DYNAMIC);
@@ -87,7 +88,9 @@ sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg)
assert(reg > 0);
struct sqlite3 *db = sqlite3VdbeDb(v);
uint32_t field_count = def->field_count;
- char *colls_type = (char *) sqlite3DbMallocZero(db, field_count + 1);
+ size_t sz = (field_count + 1) * sizeof(enum field_type);
+ enum field_type *colls_type =
+ (enum field_type *) sqlite3DbMallocZero(db, sz);
if (colls_type == NULL)
return;
for (uint32_t i = 0; i < field_count; ++i) {
@@ -104,8 +107,9 @@ sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg)
FIELD_TYPE_INTEGER);
}
}
- sqlite3VdbeAddOp4(v, OP_ApplyType, reg, field_count, 0, colls_type,
- P4_DYNAMIC);
+ colls_type[field_count] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, reg, field_count, 0,
+ (char *)colls_type, P4_DYNAMIC);
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 0ee40093f..1cd4662cf 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1202,11 +1202,11 @@ selectInnerLoop(Parse * pParse, /* The parser context */
int r1 = sqlite3GetTempReg(pParse);
assert(sqlite3Strlen30(pDest->zAffSdst) ==
(unsigned int)nResultCol);
- char *type_str =
+ enum field_type *types =
sql_affinity_str_to_field_type_str(
pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
- nResultCol, r1, type_str,
+ nResultCol, r1, (char *)types,
@@ -1630,10 +1630,11 @@ generateSortTail(Parse * pParse, /* Parsing context */
assert((unsigned int)nColumn ==
sqlite3Strlen30(pDest->zAffSdst));
- const char *type_str =
+ enum field_type *types =
sql_affinity_str_to_field_type_str(pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
- regTupleid, type_str, P4_DYNAMIC);
+ regTupleid, (char *)types,
+ P4_DYNAMIC);
@@ -3068,10 +3069,11 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
int r1;
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
- const char *type_str =
+ enum field_type *types =
sql_affinity_str_to_field_type_str(dest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
- in->nSdst, r1, type_str, P4_DYNAMIC);
+ in->nSdst, r1, (char *)types,
+ P4_DYNAMIC);
diff --git a/src/box/sql/update.c b/src/box/sql/update.c
index 41eae1550..fd74817ea 100644
--- a/src/box/sql/update.c
+++ b/src/box/sql/update.c
@@ -274,10 +274,12 @@ sqlite3Update(Parse * pParse, /* The parser context */
nKey = pk_part_count;
regKey = iPk;
} else {
- char *type_str = is_view ? NULL :
- sql_index_type_str(pParse->db, pPk->def);
+ enum field_type *types = is_view ? NULL :
+ sql_index_type_str(pParse->db,
+ pPk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, pk_part_count,
- regKey, type_str, is_view ? 0 : P4_DYNAMIC);
+ regKey, (char *) types,
+ is_view ? 0 : P4_DYNAMIC);
>> diff --git a/src/box/sql/build.c b/src/box/sql/build.c
>> index e51e2db2a..b3f98c317 100644
>> --- a/src/box/sql/build.c
>> +++ b/src/box/sql/build.c
>> @@ -520,6 +520,19 @@ sql_field_type_to_affinity(enum field_type field_type)
>> }
>> }
>> +char *
>
> 2. So not char *, but enum field_type *. Ok?
Yep, see below.
>
>> +sql_affinity_str_to_field_type_str(const char *affinity_str)
>> +{
>> + if (affinity_str == NULL)
>> + return NULL;
>> + size_t len = strlen(affinity_str) + 1;
>> + char *type_str = (char *) sqlite3DbMallocRaw(sql_get(), len);
>> + for (uint32_t i = 0; i < len - 1; ++i)
>> + type_str[i] = sql_affinity_to_field_type(affinity_str[i]);
>> + type_str[len - 1] = '\0';
>
> 2. Instead of 0 you can use field_type_MAX terminator, if we will move
> to enum field_type[].
Ok:
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index b3f98c317..70abe5e57 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -520,17 +520,18 @@ sql_field_type_to_affinity(enum field_type field_type)
}
}
-char *
+enum field_type *
sql_affinity_str_to_field_type_str(const char *affinity_str)
{
if (affinity_str == NULL)
return NULL;
- size_t len = strlen(affinity_str) + 1;
- char *type_str = (char *) sqlite3DbMallocRaw(sql_get(), len);
+ size_t len = (strlen(affinity_str) + 1) * sizeof(enum field_type);
+ enum field_type *types =
+ (enum field_type *) sqlite3DbMallocRaw(sql_get(), len);
for (uint32_t i = 0; i < len - 1; ++i)
- type_str[i] = sql_affinity_to_field_type(affinity_str[i]);
- type_str[len - 1] = '\0';
- return type_str;
+ types[i] = sql_affinity_to_field_type(affinity_str[i]);
+ types[len - 1] = field_type_MAX;
+ return types;
}
/*
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index fd36e2786..b122b872e 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -68,16 +68,17 @@ sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
return aff;
}
-char *
+enum field_type *
sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def)
{
uint32_t column_count = idx_def->key_def->part_count;
- char *types = (char *) sqlite3DbMallocRaw(db, column_count + 1);
+ uint32_t sz = (column_count + 1) * sizeof(enum field_type);
+ enum field_type *types = (enum field_type *) sqlite3DbMallocRaw(db, sz);
if (types == NULL)
return NULL;
for (uint32_t i = 0; i < column_count; i++)
types[i] = idx_def->key_def->parts[i].type;
- types[column_count] = '\0';
+ types[column_count] = field_type_MAX;
return types;
}
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 690fa6431..c8adc9ffe 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3439,7 +3439,7 @@ sql_affinity_to_field_type(enum affinity_type affinity);
enum affinity_type
sql_field_type_to_affinity(enum field_type field_type);
-char *
+enum field_type *
sql_affinity_str_to_field_type_str(const char *affinity_str);
@@ -4239,7 +4239,7 @@ sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
struct index_def *idx_def);
/** Return string consisting of fields types of given index. */
-char *
+enum field_type *
sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 369fb4b79..61d73b676 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2813,12 +2813,12 @@ case OP_Column: {
* memory cell in the range.
*/
case OP_ApplyType: {
- const char *type_str = pOp->p4.z;
+ enum field_type *type_str = (enum field_type *)pOp->p4.z;
assert(type_str != NULL);
- assert(type_str[pOp->p2] == '\0');
+ assert(type_str[pOp->p2] == field_type_MAX);
pIn1 = &aMem[pOp->p1];
- char type;
- while((type = *(type_str++)) != '\0') {
+ enum field_type type;
+ while((type = *(type_str++)) != field_type_MAX) {
assert(pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)]);
assert(memIsValid(pIn1));
if (mem_apply_type(pIn1, type) != 0) {
@@ -2855,7 +2855,6 @@ case OP_MakeRecord: {
Mem *pData0; /* First field to be combined into the record */
Mem MAYBE_UNUSED *pLast; /* Last field of the record */
int nField; /* Number of fields in the record */
- char *zAffinity; /* The affinity string for the record */
u8 bIsEphemeral;
/* Assuming the record contains N fields, the record format looks
@@ -2874,7 +2873,7 @@ case OP_MakeRecord: {
* of the record to data0.
*/
nField = pOp->p1;
- zAffinity = pOp->p4.z;
+ enum field_type *types = (enum field_type *)pOp->p4.z;
bIsEphemeral = pOp->p5;
assert(nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem+1 - p->nCursor)+1);
pData0 = &aMem[nField];
@@ -2889,11 +2888,11 @@ case OP_MakeRecord: {
/* Apply the requested affinity to all inputs
*/
assert(pData0<=pLast);
- if (zAffinity) {
+ if (types != NULL) {
pRec = pData0;
- do{
- mem_apply_type(pRec++, *(zAffinity++));
- }while( zAffinity[0]);
+ do {
+ mem_apply_type(pRec++, *(types++));
+ } while(types[0] != field_type_MAX);
}
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index aae5d6617..efbc91cf8 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -397,8 +397,11 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
}
if (n > 0) {
- const char *type_str = sql_affinity_str_to_field_type_str(zAff);
- sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, type_str, n);
+ enum field_type *types =
+ sql_affinity_str_to_field_type_str(zAff);
+ types[n] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *)types,
+ P4_DYNAMIC);
>
>> + return type_str;
>> +}
>> +
>> /*
>> * Add a new column to the table currently being constructed.
>> *
>> diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
>> index f9c42fdec..7b0d6b2fd 100644
>> --- a/src/box/sql/delete.c
>> +++ b/src/box/sql/delete.c
>> @@ -332,12 +332,12 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
>> */
>> key_len = 0;
>> struct index *pk = space_index(space, 0);
>> - const char *zAff = is_view ? NULL :
>> - sql_space_index_affinity_str(parse->db,
>> - space->def,
>> - pk->def);
>> + char *type_str = is_view ? NULL :
>> + sql_index_type_str(parse->db,
>> + pk->def);
>> sqlite3VdbeAddOp4(v, OP_MakeRecord, reg_pk, pk_len,
>> - reg_key, zAff, pk_len);
>> + reg_key, type_str, is_view ? 0 :
>> + P4_DYNAMIC);
>
> 3. How did it work before your patch? Looks like it was a leak. Before the
> patch, pk_len was passed instead of STATIC/DYNAMIC.
Yep, type_str was copied inside vdbeChangeP4Full(), but zAff itself
would be never freed. Now it works as should.
>
>> /* Set flag to save memory allocating one
>> * by malloc.
>> */
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index 917e6e30b..c823c5a06 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -2858,8 +2858,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>> jmpIfDynamic = -1;
>> }
>> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
>> + char type =
>> + sql_affinity_to_field_type(affinity);
>> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
>> - 1, r2, &affinity, 1);
>> + 1, r2, &type,
>> + 1);
>
> 4. I do not understand. Is p4type of sqlite3VdbeAddOp4 of OP_MakeRecord
> a length of an affinity string, or a type of its allocation? Or both?
Both. p4type > 0 means that p4type bytes are copied to freshly allocated memory
and P4 set to DYNAMIC. sqlite3DbStrNDup() in vdbeChangeP4Full() reserves
one more byte for NULL termination. After replacing char* with field_type:
+++ b/src/box/sql/expr.c
@@ -2858,11 +2858,12 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
jmpIfDynamic = -1;
}
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
- char type =
+ enum field_type type =
sql_affinity_to_field_type(affinity);
+ enum field_type types[2] = {type, field_type_MAX};
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
- 1, r2, &type,
- 1);
+ 1, r2, (char *)types,
+ sizeof(types));
>
> Also, I wonder how does this code works:
>
> if (zAffinity) {
> pRec = pData0;
> do{
> mem_apply_type(pRec++, *(zAffinity++));
> }while( zAffinity[0]);
> }
>
> in OP_MakeRecord. It assumes, that zAffinity is a null-terminated string,
> but in the code above you pass one char, without zero-termination.
It is OK, string is copied and null terminated, see explanation above.
Nevertheless, this code has been reworked, since now we use
field_type_MAX as a termination sign.
>> sqlite3ExprCacheAffinityChange(pParse,
>> r3, 1);
>> sqlite3VdbeAddOp2(v, OP_IdxInsert, r2,
>> @@ -3172,7 +3175,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
>> * of the RHS using the LHS as a probe. If found, the result is
>> * true.
>> */
>> - sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
>> + char *type_str = sql_affinity_str_to_field_type_str(zAff);
>> + sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, type_str, nVector);
>
> 5. type_str is dynamically allocated, nVector (== p4type) is > 0. But freeP4
> function in vdbeaux.c does not know how to free p4type > 0. So it is definitely
> a leak. Please, validate all places where dynamic type string is created.
Yep, it is similar to the first one in this patch, but I’ve missed this place.
Should be P4_DYNAMIC. Moreover, now we use field_type_MAX to
indicate end of “string”, so we should cut it to follow original behaviour.
Fixed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index c823c5a06..2be083fb0 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3175,8 +3177,10 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- char *type_str = sql_affinity_str_to_field_type_str(zAff);
- sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, type_str, nVector);
+ enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
+ types[nVector] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
+ P4_DYNAMIC);
>
>> if (destIfFalse == destIfNull) {
>> /* Combine Step 3 and Step 5 into a single opcode */
>> sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 4345af24e..369fb4b79 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -306,32 +306,35 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
>> }
>> /**
>> - * Processing is determine by the affinity parameter:
>> + * Processing is determine by the field type parameter:
>> *
>> - * AFFINITY_INTEGER:
>> - * AFFINITY_REAL:
>> - * Try to convert mem to an integer representation or a
>> - * floating-point representation if an integer representation
>> - * is not possible. Note that the integer representation is
>> - * always preferred, even if the affinity is REAL, because
>> - * an integer representation is more space efficient on disk.
>> + * INTEGER:
>> + * If memory holds floating point value and it can be
>> + * converted without loss (2.0 - > 2), it's type is
>> + * changed to INT. Otherwise, simply return success status.
>> *
>> - * AFFINITY_TEXT:
>> - * Convert mem to a text representation.
>> + * NUMBER:
>> + * If memory holds INT or floating point value,
>> + * no actions take place.
>> *
>> - * AFFINITY_BLOB:
>> - * No-op. mem is unchanged.
>> + * STRING:
>> + * Convert mem to a string representation.
>> *
>> - * @param record The value to apply affinity to.
>> - * @param affinity The affinity to be applied.
>> + * SCALAR:
>> + * Mem is unchanged, but flat is set to BLOB.
>
> 6. flat -> flag?
Fixed:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 369fb4b79..34365edd0 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -321,7 +321,7 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
* Convert mem to a string representation.
*
* SCALAR:
- * Mem is unchanged, but flat is set to BLOB.
+ * Mem is unchanged, but flag is set to BLOB.
*
>
>> + *
>> + * @param record The value to apply type to.
>> + * @param type_t The type to be applied.
>> */
>> static int
>> -mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
>> +mem_apply_type(struct Mem *record, enum field_type f_type)
>> {
>> if ((record->flags & MEM_Null) != 0)
>> return 0;
>> - switch (affinity) {
>> - case AFFINITY_INTEGER:
>> + assert(f_type < field_type_MAX);
>
> 7. Double white-space.
Fixed:
@@ -331,7 +331,7 @@ mem_apply_type(struct Mem *record, enum field_type f_type)
{
if ((record->flags & MEM_Null) != 0)
return 0;
- assert(f_type < field_type_MAX);
+ assert(f_type < field_type_MAX);
>
>> + switch (f_type) {
>> + case FIELD_TYPE_INTEGER:
>> + case FIELD_TYPE_UNSIGNED:
>> if ((record->flags & MEM_Int) == MEM_Int)
>> return 0;
>> if ((record->flags & MEM_Real) == MEM_Real) {
>> diff --git a/src/box/sql/where.c b/src/box/sql/where.c
>> index 571b5af78..539296079 100644
>> --- a/src/box/sql/where.c
>> +++ b/src/box/sql/where.c
>> @@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
>> int nLower = -1;
>> int nUpper = index->def->opts.stat->sample_count + 1;
>> int rc = SQLITE_OK;
>> - u8 aff = sql_space_index_part_affinity(space->def, p, nEq);
>> + u8 aff = p->key_def->parts[nEq].type;
>
> 8. Why? Below in this function aff is used as affinity, not type.
Am I missing smth?
sqlite3Stat4ValueFromExpr -> stat4ValueFromExpr -> sqlite3ValueApplyAffinity -> mem_apply_type
And mem_apply_type operates on field_type, not affinity.
Also, renamed variable.
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 539296079..6b8999dd9 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
int nLower = -1;
int nUpper = index->def->opts.stat->sample_count + 1;
int rc = SQLITE_OK;
- u8 aff = p->key_def->parts[nEq].type;
+ enum field_type type = p->key_def->parts[nEq].type;
sqlite3_value *p1 = 0; /* Value extracted from pLower */
sqlite3_value *p2 = 0; /* Value extracted from pUpper */
@@ -1209,12 +1209,12 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
struct coll *coll = p->key_def->parts[nEq].coll;
if (pLower) {
rc = sqlite3Stat4ValueFromExpr(pParse, pLower->pExpr->pRight,
- aff, &p1);
+ type, &p1);
nLower = 0;
}
if (pUpper && rc == SQLITE_OK) {
rc = sqlite3Stat4ValueFromExpr(pParse, pUpper->pExpr->pRight,
- aff, &p2);
+ type, &p2);
nUpper = p2 ? 0 : index->def->opts.stat->sample_count;
Whole patch:
Author: Nikita Pettik <korablev@tarantool.org>
Date: Fri Dec 21 13:23:07 2018 +0200
sql: replace affinity with field type for VDBE runtime
This stage of affinity removal requires introducing of auxiliary
intermediate function to convert array of affinity values to field type
values. The rest of job done in this commit is a straightforward
refactoring.
Part of #3698
diff --git a/src/box/sql/analyze.c b/src/box/sql/analyze.c
index 51c63fa7a..982eaa974 100644
--- a/src/box/sql/analyze.c
+++ b/src/box/sql/analyze.c
@@ -993,9 +993,12 @@ vdbe_emit_analyze_space(struct Parse *parse, struct space *space)
sqlite3VdbeAddOp2(v, OP_Next, idx_cursor, next_row_addr);
/* Add the entry to the stat1 table. */
callStatGet(v, stat4_reg, STAT_GET_STAT1, stat1_reg);
- assert("BBB"[0] == AFFINITY_TEXT);
- sqlite3VdbeAddOp4(v, OP_MakeRecord, tab_name_reg, 3, tmp_reg,
- "BBB", 0);
+ enum field_type types[4] = { FIELD_TYPE_STRING,
+ FIELD_TYPE_STRING,
+ FIELD_TYPE_STRING,
+ field_type_MAX };
+ sqlite3VdbeAddOp4(v, OP_MakeRecord, tab_name_reg, 4, tmp_reg,
+ (char *)types, sizeof(types));
sqlite3VdbeAddOp4(v, OP_IdxInsert, tmp_reg, 0, 0,
(char *)stat1, P4_SPACEPTR);
/* Add the entries to the stat4 table. */
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index e51e2db2a..514d0ca9d 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -520,6 +520,21 @@ sql_field_type_to_affinity(enum field_type field_type)
}
}
+enum field_type *
+sql_affinity_str_to_field_type_str(const char *affinity_str)
+{
+ if (affinity_str == NULL)
+ return NULL;
+ size_t len = strlen(affinity_str) + 1;
+ size_t sz = len * sizeof(enum field_type);
+ enum field_type *types =
+ (enum field_type *) sqlite3DbMallocRaw(sql_get(), sz);
+ for (uint32_t i = 0; i < len - 1; ++i)
+ types[i] = sql_affinity_to_field_type(affinity_str[i]);
+ types[len - 1] = field_type_MAX;
+ return types;
+}
+
/*
* Add a new column to the table currently being constructed.
*
diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
index f9c42fdec..ca6c49373 100644
--- a/src/box/sql/delete.c
+++ b/src/box/sql/delete.c
@@ -332,12 +332,12 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
*/
key_len = 0;
struct index *pk = space_index(space, 0);
- const char *zAff = is_view ? NULL :
- sql_space_index_affinity_str(parse->db,
- space->def,
- pk->def);
+ enum field_type *types = is_view ? NULL :
+ sql_index_type_str(parse->db,
+ pk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, reg_pk, pk_len,
- reg_key, zAff, pk_len);
+ reg_key, (char *)types, is_view ? 0 :
+ P4_DYNAMIC);
/* Set flag to save memory allocating one
* by malloc.
*/
@@ -592,13 +592,13 @@ sql_generate_index_key(struct Parse *parse, struct index *index, int cursor,
* is an integer, then it might be stored in the
* table as an integer (using a compact
* representation) then converted to REAL by an
- * OP_RealAffinity opcode. But we are getting
+ * OP_Realify opcode. But we are getting
* ready to store this value back into an index,
* where it should be converted by to INTEGER
- * again. So omit the OP_RealAffinity opcode if
+ * again. So omit the OP_Realify opcode if
* it is present
*/
- sqlite3VdbeDeletePriorOpcode(v, OP_RealAffinity);
+ sqlite3VdbeDeletePriorOpcode(v, OP_Realify);
}
if (reg_out != 0)
sqlite3VdbeAddOp3(v, OP_MakeRecord, reg_base, col_cnt, reg_out);
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 0a80ca622..ca39faf51 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -2146,10 +2146,10 @@ sqlite3ExprCanBeNull(const Expr * p)
/*
* Return TRUE if the given expression is a constant which would be
- * unchanged by OP_Affinity with the affinity given in the second
+ * unchanged by OP_ApplyType with the type given in the second
* argument.
*
- * This routine is used to determine if the OP_Affinity operation
+ * This routine is used to determine if the OP_ApplyType operation
* can be omitted. When in doubt return FALSE. A false negative
* is harmless. A false positive, however, can result in the wrong
* answer.
@@ -2858,8 +2858,13 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
jmpIfDynamic = -1;
}
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
+ enum field_type type =
+ sql_affinity_to_field_type(affinity);
+ enum field_type types[2] =
+ { type, field_type_MAX };
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
- 1, r2, &affinity, 1);
+ 1, r2, (char *)types,
+ sizeof(types));
sqlite3ExprCacheAffinityChange(pParse,
r3, 1);
sqlite3VdbeAddOp2(v, OP_IdxInsert, r2,
@@ -3172,7 +3177,10 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
+ enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
+ types[nVector] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
+ P4_DYNAMIC);
if (destIfFalse == destIfNull) {
/* Combine Step 3 and Step 5 into a single opcode */
sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,
@@ -3700,7 +3708,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
pCol->iSorterColumn, target);
if (pCol->space_def->fields[pExpr->iAgg].type ==
FIELD_TYPE_NUMBER) {
- sqlite3VdbeAddOp1(v, OP_RealAffinity,
+ sqlite3VdbeAddOp1(v, OP_Realify,
target);
}
return target;
@@ -3800,7 +3808,8 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
sqlite3VdbeAddOp2(v, OP_SCopy, inReg, target);
inReg = target;
}
- sqlite3VdbeAddOp2(v, OP_Cast, target, pExpr->affinity);
+ sqlite3VdbeAddOp2(v, OP_Cast, target,
+ sql_affinity_to_field_type(pExpr->affinity));
testcase(usedAsColumnCache(pParse, inReg, inReg));
sqlite3ExprCacheAffinityChange(pParse, inReg, 1);
return inReg;
@@ -4236,14 +4245,14 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
#ifndef SQLITE_OMIT_FLOATING_POINT
/* If the column has REAL affinity, it may currently be stored as an
- * integer. Use OP_RealAffinity to make sure it is really real.
+ * integer. Use OP_Realify to make sure it is really real.
*
* EVIDENCE-OF: R-60985-57662 SQLite will convert the value back to
* floating point when extracting it from the record.
*/
if (pExpr->iColumn >= 0 && def->fields[
pExpr->iColumn].affinity == AFFINITY_REAL) {
- sqlite3VdbeAddOp1(v, OP_RealAffinity, target);
+ sqlite3VdbeAddOp1(v, OP_Realify, target);
}
#endif
break;
diff --git a/src/box/sql/fkey.c b/src/box/sql/fkey.c
index 4e3270f0c..8aa8ea01e 100644
--- a/src/box/sql/fkey.c
+++ b/src/box/sql/fkey.c
@@ -256,9 +256,8 @@ fkey_lookup_parent(struct Parse *parse_context, struct space *parent,
struct index *idx = space_index(parent, referenced_idx);
assert(idx != NULL);
sqlite3VdbeAddOp4(v, OP_MakeRecord, temp_regs, field_count, rec_reg,
- sql_space_index_affinity_str(parse_context->db,
- parent->def, idx->def),
- P4_DYNAMIC);
+ (char *) sql_index_type_str(parse_context->db,
+ idx->def), P4_DYNAMIC);
sqlite3VdbeAddOp4Int(v, OP_Found, cursor, ok_label, rec_reg, 0);
sqlite3ReleaseTempReg(parse_context, rec_reg);
sqlite3ReleaseTempRange(parse_context, temp_regs, field_count);
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index 6b76bb6da..b122b872e 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -68,17 +68,33 @@ sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
return aff;
}
+enum field_type *
+sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def)
+{
+ uint32_t column_count = idx_def->key_def->part_count;
+ uint32_t sz = (column_count + 1) * sizeof(enum field_type);
+ enum field_type *types = (enum field_type *) sqlite3DbMallocRaw(db, sz);
+ if (types == NULL)
+ return NULL;
+ for (uint32_t i = 0; i < column_count; i++)
+ types[i] = idx_def->key_def->parts[i].type;
+ types[column_count] = field_type_MAX;
+ return types;
+}
+
void
-sql_emit_table_affinity(struct Vdbe *v, struct space_def *def, int reg)
+sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg)
{
assert(reg > 0);
struct sqlite3 *db = sqlite3VdbeDb(v);
uint32_t field_count = def->field_count;
- char *colls_aff = (char *) sqlite3DbMallocZero(db, field_count + 1);
- if (colls_aff == NULL)
+ size_t sz = (field_count + 1) * sizeof(enum field_type);
+ enum field_type *colls_type =
+ (enum field_type *) sqlite3DbMallocZero(db, sz);
+ if (colls_type == NULL)
return;
for (uint32_t i = 0; i < field_count; ++i) {
- colls_aff[i] = def->fields[i].affinity;
+ colls_type[i] = def->fields[i].type;
/*
* Force INTEGER type to handle queries like:
* CREATE TABLE t1 (id INT PRIMARY KEY);
@@ -86,13 +102,14 @@ sql_emit_table_affinity(struct Vdbe *v, struct space_def *def, int reg)
*
* In this case 1.123 should be truncated to 1.
*/
- if (colls_aff[i] == AFFINITY_INTEGER) {
+ if (colls_type[i] == FIELD_TYPE_INTEGER) {
sqlite3VdbeAddOp2(v, OP_Cast, reg + i,
- AFFINITY_INTEGER);
+ FIELD_TYPE_INTEGER);
}
}
- sqlite3VdbeAddOp4(v, OP_Affinity, reg, field_count, 0, colls_aff,
- P4_DYNAMIC);
+ colls_type[field_count] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, reg, field_count, 0,
+ (char *)colls_type, P4_DYNAMIC);
}
/**
@@ -616,7 +633,7 @@ sqlite3Insert(Parse * pParse, /* Parser context */
* table column affinities.
*/
if (!is_view)
- sql_emit_table_affinity(v, pTab->def, regCols + 1);
+ sql_emit_table_types(v, pTab->def, regCols + 1);
/* Fire BEFORE or INSTEAD OF triggers */
vdbe_code_row_trigger(pParse, trigger, TK_INSERT, 0,
@@ -964,7 +981,7 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct Table *tab,
sqlite3VdbeResolveLabel(v, all_ok);
}
}
- sql_emit_table_affinity(v, tab->def, new_tuple_reg);
+ sql_emit_table_types(v, tab->def, new_tuple_reg);
/*
* If PK is marked as INTEGER, use it as strict type,
* not as affinity. Emit code for type checking.
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index fa0fea6c8..4b6983842 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1202,9 +1202,12 @@ selectInnerLoop(Parse * pParse, /* The parser context */
int r1 = sqlite3GetTempReg(pParse);
assert(sqlite3Strlen30(pDest->zAffSdst) ==
(unsigned int)nResultCol);
+ enum field_type *types =
+ sql_affinity_str_to_field_type_str(
+ pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
- nResultCol, r1,
- pDest->zAffSdst, nResultCol);
+ nResultCol, r1, (char *)types,
+ P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(pParse,
regResult,
nResultCol);
@@ -1626,8 +1629,12 @@ generateSortTail(Parse * pParse, /* Parsing context */
case SRT_Set:{
assert((unsigned int)nColumn ==
sqlite3Strlen30(pDest->zAffSdst));
+
+ enum field_type *types =
+ sql_affinity_str_to_field_type_str(pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
- regTupleid, pDest->zAffSdst, nColumn);
+ regTupleid, (char *)types,
+ P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(pParse, regRow, nColumn);
sqlite3VdbeAddOp2(v, OP_IdxInsert, regTupleid, pDest->reg_eph);
break;
@@ -3062,9 +3069,11 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
int r1;
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
+ enum field_type *types =
+ sql_affinity_str_to_field_type_str(dest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
- in->nSdst, r1, dest->zAffSdst,
- in->nSdst);
+ in->nSdst, r1, (char *)types,
+ P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(parse, in->iSdst,
in->nSdst);
sqlite3VdbeAddOp2(v, OP_IdxInsert, r1, dest->reg_eph);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 2a7223fff..579f68fed 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3439,6 +3439,9 @@ sql_affinity_to_field_type(enum affinity_type affinity);
enum affinity_type
sql_field_type_to_affinity(enum field_type field_type);
+enum field_type *
+sql_affinity_str_to_field_type_str(const char *affinity_str);
+
/**
* Compile view, i.e. create struct Select from
* 'CREATE VIEW...' string, and assign cursors to each table from
@@ -4235,16 +4238,20 @@ char *
sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
struct index_def *idx_def);
+/** Return string consisting of fields types of given index. */
+enum field_type *
+sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
+
/**
- * Code an OP_Affinity opcode that will set affinities
+ * Code an OP_ApplyType opcode that will force types
* for given range of register starting from @reg.
*
* @param v VDBE.
* @param def Definition of table to be used.
- * @param reg Register where affinities will be placed.
+ * @param reg Register where types will be placed.
*/
void
-sql_emit_table_affinity(struct Vdbe *v, struct space_def *def, int reg);
+sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
/**
* Return superposition of two affinities.
diff --git a/src/box/sql/update.c b/src/box/sql/update.c
index 0e2d0fde8..fd74817ea 100644
--- a/src/box/sql/update.c
+++ b/src/box/sql/update.c
@@ -44,7 +44,7 @@ sqlite3ColumnDefault(Vdbe *v, struct space_def *def, int i, int ireg)
assert(def != 0);
if (!def->opts.is_view) {
sqlite3_value *pValue = 0;
- char affinity = def->fields[i].affinity;
+ enum field_type type = def->fields[i].type;
VdbeComment((v, "%s.%s", def->name, def->fields[i].name));
assert(i < (int)def->field_count);
@@ -52,14 +52,14 @@ sqlite3ColumnDefault(Vdbe *v, struct space_def *def, int i, int ireg)
assert(def->fields != NULL && i < (int)def->field_count);
if (def->fields != NULL)
expr = def->fields[i].default_value_expr;
- sqlite3ValueFromExpr(sqlite3VdbeDb(v), expr, affinity,
+ sqlite3ValueFromExpr(sqlite3VdbeDb(v), expr, type,
&pValue);
if (pValue) {
sqlite3VdbeAppendP4(v, pValue, P4_MEM);
}
#ifndef SQLITE_OMIT_FLOATING_POINT
- if (affinity == AFFINITY_REAL) {
- sqlite3VdbeAddOp1(v, OP_RealAffinity, ireg);
+ if (type == FIELD_TYPE_NUMBER) {
+ sqlite3VdbeAddOp1(v, OP_Realify, ireg);
}
#endif
}
@@ -274,11 +274,12 @@ sqlite3Update(Parse * pParse, /* The parser context */
nKey = pk_part_count;
regKey = iPk;
} else {
- const char *zAff = is_view ? 0 :
- sql_space_index_affinity_str(pParse->db, def,
- pPk->def);
+ enum field_type *types = is_view ? NULL :
+ sql_index_type_str(pParse->db,
+ pPk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, pk_part_count,
- regKey, zAff, pk_part_count);
+ regKey, (char *) types,
+ is_view ? 0 : P4_DYNAMIC);
/*
* Set flag to save memory allocating one by
* malloc.
@@ -390,7 +391,7 @@ sqlite3Update(Parse * pParse, /* The parser context */
* verified. One could argue that this is wrong.
*/
if (tmask & TRIGGER_BEFORE) {
- sql_emit_table_affinity(v, pTab->def, regNew);
+ sql_emit_table_types(v, pTab->def, regNew);
vdbe_code_row_trigger(pParse, trigger, TK_UPDATE, pChanges,
TRIGGER_BEFORE, pTab, regOldPk,
on_error, labelContinue);
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 4345af24e..61d73b676 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -306,32 +306,35 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
}
/**
- * Processing is determine by the affinity parameter:
+ * Processing is determine by the field type parameter:
*
- * AFFINITY_INTEGER:
- * AFFINITY_REAL:
- * Try to convert mem to an integer representation or a
- * floating-point representation if an integer representation
- * is not possible. Note that the integer representation is
- * always preferred, even if the affinity is REAL, because
- * an integer representation is more space efficient on disk.
+ * INTEGER:
+ * If memory holds floating point value and it can be
+ * converted without loss (2.0 - > 2), it's type is
+ * changed to INT. Otherwise, simply return success status.
*
- * AFFINITY_TEXT:
- * Convert mem to a text representation.
+ * NUMBER:
+ * If memory holds INT or floating point value,
+ * no actions take place.
*
- * AFFINITY_BLOB:
- * No-op. mem is unchanged.
+ * STRING:
+ * Convert mem to a string representation.
*
- * @param record The value to apply affinity to.
- * @param affinity The affinity to be applied.
+ * SCALAR:
+ * Mem is unchanged, but flag is set to BLOB.
+ *
+ * @param record The value to apply type to.
+ * @param type_t The type to be applied.
*/
static int
-mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
+mem_apply_type(struct Mem *record, enum field_type f_type)
{
if ((record->flags & MEM_Null) != 0)
return 0;
- switch (affinity) {
- case AFFINITY_INTEGER:
+ assert(f_type < field_type_MAX);
+ switch (f_type) {
+ case FIELD_TYPE_INTEGER:
+ case FIELD_TYPE_UNSIGNED:
if ((record->flags & MEM_Int) == MEM_Int)
return 0;
if ((record->flags & MEM_Real) == MEM_Real) {
@@ -343,11 +346,11 @@ mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
return 0;
}
return sqlite3VdbeMemIntegerify(record, false);
- case AFFINITY_REAL:
+ case FIELD_TYPE_NUMBER:
if ((record->flags & (MEM_Real | MEM_Int)) != 0)
return 0;
return sqlite3VdbeMemRealify(record);
- case AFFINITY_TEXT:
+ case FIELD_TYPE_STRING:
/*
* Only attempt the conversion to TEXT if there is
* an integer or real representation (BLOB and
@@ -359,7 +362,7 @@ mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
}
record->flags &= ~(MEM_Real | MEM_Int);
return 0;
- case AFFINITY_BLOB:
+ case FIELD_TYPE_SCALAR:
if (record->flags & (MEM_Str | MEM_Blob))
record->flags |= MEM_Blob;
return 0;
@@ -385,7 +388,7 @@ int sqlite3_value_numeric_type(sqlite3_value *pVal) {
}
/*
- * Exported version of mem_apply_affinity(). This one works on sqlite3_value*,
+ * Exported version of mem_apply_type(). This one works on sqlite3_value*,
* not the internal Mem* type.
*/
void
@@ -393,7 +396,7 @@ sqlite3ValueApplyAffinity(
sqlite3_value *pVal,
u8 affinity)
{
- mem_apply_affinity((Mem *) pVal, affinity);
+ mem_apply_type((Mem *) pVal, affinity);
}
/*
@@ -1946,7 +1949,7 @@ case OP_AddImm: { /* in1 */
case OP_MustBeInt: { /* jump, in1 */
pIn1 = &aMem[pOp->p1];
if ((pIn1->flags & MEM_Int)==0) {
- mem_apply_affinity(pIn1, AFFINITY_INTEGER);
+ mem_apply_type(pIn1, FIELD_TYPE_INTEGER);
VdbeBranchTaken((pIn1->flags&MEM_Int)==0, 2);
if ((pIn1->flags & MEM_Int)==0) {
if (pOp->p2==0) {
@@ -1962,16 +1965,16 @@ case OP_MustBeInt: { /* jump, in1 */
}
#ifndef SQLITE_OMIT_FLOATING_POINT
-/* Opcode: RealAffinity P1 * * * *
+/* Opcode: Realify P1 * * * *
*
* If register P1 holds an integer convert it to a real value.
*
* This opcode is used when extracting information from a column that
- * has REAL affinity. Such column values may still be stored as
+ * has float type. Such column values may still be stored as
* integers, for space efficiency, but after extraction we want them
* to have only a real value.
*/
-case OP_RealAffinity: { /* in1 */
+case OP_Realify: { /* in1 */
pIn1 = &aMem[pOp->p1];
if (pIn1->flags & MEM_Int) {
sqlite3VdbeMemRealify(pIn1);
@@ -1982,7 +1985,7 @@ case OP_RealAffinity: { /* in1 */
#ifndef SQLITE_OMIT_CAST
/* Opcode: Cast P1 P2 * * *
- * Synopsis: affinity(r[P1])
+ * Synopsis: type(r[P1])
*
* Force the value in register P1 to be the type defined by P2.
*
@@ -1997,11 +2000,6 @@ case OP_RealAffinity: { /* in1 */
* A NULL value is not changed by this routine. It remains NULL.
*/
case OP_Cast: { /* in1 */
- assert(pOp->p2>=AFFINITY_BLOB && pOp->p2<=AFFINITY_REAL);
- testcase( pOp->p2==AFFINITY_TEXT);
- testcase( pOp->p2==AFFINITY_BLOB);
- testcase( pOp->p2==AFFINITY_INTEGER);
- testcase( pOp->p2==AFFINITY_REAL);
pIn1 = &aMem[pOp->p1];
rc = ExpandBlob(pIn1);
if (rc != 0)
@@ -2011,7 +2009,7 @@ case OP_Cast: { /* in1 */
if (rc == 0)
break;
diag_set(ClientError, ER_SQL_TYPE_MISMATCH, sqlite3_value_text(pIn1),
- affinity_type_str(pOp->p2));
+ field_type_strs[pOp->p2]);
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
@@ -2225,7 +2223,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
default: res2 = res>=0; break;
}
- /* Undo any changes made by mem_apply_affinity() to the input registers. */
+ /* Undo any changes made by mem_apply_type() to the input registers. */
assert((pIn1->flags & MEM_Dyn) == (flags1 & MEM_Dyn));
pIn1->flags = flags1;
assert((pIn3->flags & MEM_Dyn) == (flags3 & MEM_Dyn));
@@ -2814,21 +2812,19 @@ case OP_Column: {
* string indicates the column affinity that should be used for the nth
* memory cell in the range.
*/
-case OP_Affinity: {
- const char *zAffinity; /* The affinity to be applied */
- char cAff; /* A single character of affinity */
-
- zAffinity = pOp->p4.z;
- assert(zAffinity!=0);
- assert(zAffinity[pOp->p2]==0);
+case OP_ApplyType: {
+ enum field_type *type_str = (enum field_type *)pOp->p4.z;
+ assert(type_str != NULL);
+ assert(type_str[pOp->p2] == field_type_MAX);
pIn1 = &aMem[pOp->p1];
- while( (cAff = *(zAffinity++))!=0) {
+ enum field_type type;
+ while((type = *(type_str++)) != field_type_MAX) {
assert(pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)]);
assert(memIsValid(pIn1));
- if (mem_apply_affinity(pIn1, cAff) != 0) {
+ if (mem_apply_type(pIn1, type) != 0) {
diag_set(ClientError, ER_SQL_TYPE_MISMATCH,
sqlite3_value_text(pIn1),
- affinity_type_str(cAff));
+ field_type_strs[type]);
rc = SQL_TARANTOOL_ERROR;
goto abort_due_to_error;
}
@@ -2848,10 +2844,7 @@ case OP_Affinity: {
* string indicates the column affinity that should be used for the nth
* field of the index key.
*
- * The mapping from character to affinity is given by the AFFINITY_
- * macros defined in sqliteInt.h.
- *
- * If P4 is NULL then all index fields have the affinity BLOB.
+ * If P4 is NULL then all index fields have type SCALAR.
*
* If P5 is not NULL then record under construction is intended to be inserted
* into ephemeral space. Thus, sort of memory optimization can be performed.
@@ -2862,7 +2855,6 @@ case OP_MakeRecord: {
Mem *pData0; /* First field to be combined into the record */
Mem MAYBE_UNUSED *pLast; /* Last field of the record */
int nField; /* Number of fields in the record */
- char *zAffinity; /* The affinity string for the record */
u8 bIsEphemeral;
/* Assuming the record contains N fields, the record format looks
@@ -2881,7 +2873,7 @@ case OP_MakeRecord: {
* of the record to data0.
*/
nField = pOp->p1;
- zAffinity = pOp->p4.z;
+ enum field_type *types = (enum field_type *)pOp->p4.z;
bIsEphemeral = pOp->p5;
assert(nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem+1 - p->nCursor)+1);
pData0 = &aMem[nField];
@@ -2896,12 +2888,11 @@ case OP_MakeRecord: {
/* Apply the requested affinity to all inputs
*/
assert(pData0<=pLast);
- if (zAffinity) {
+ if (types != NULL) {
pRec = pData0;
- do{
- mem_apply_affinity(pRec++, *(zAffinity++));
- assert(zAffinity[0]==0 || pRec<=pLast);
- }while( zAffinity[0]);
+ do {
+ mem_apply_type(pRec++, *(types++));
+ } while(types[0] != field_type_MAX);
}
/* Loop through the elements that will make up the record to figure
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 50bc35b2b..879ba34d0 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -483,7 +483,7 @@ int sqlite3VdbeRealValue(Mem *, double *);
int sqlite3VdbeIntegerAffinity(Mem *);
int sqlite3VdbeMemRealify(Mem *);
int sqlite3VdbeMemNumerify(Mem *);
-int sqlite3VdbeMemCast(Mem *, u8);
+int sqlite3VdbeMemCast(Mem *, enum field_type type);
int sqlite3VdbeMemFromBtree(BtCursor *, u32, u32, Mem *);
void sqlite3VdbeMemRelease(Mem * p);
int sqlite3VdbeMemFinalize(Mem *, FuncDef *);
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index bb4d91aed..cd71641b0 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -598,11 +598,12 @@ sqlite3VdbeMemNumerify(Mem * pMem)
* used (for example) to implement the SQL "cast()" operator.
*/
int
-sqlite3VdbeMemCast(Mem * pMem, u8 aff)
+sqlite3VdbeMemCast(Mem * pMem, enum field_type type)
{
+ assert(type < field_type_MAX);
if (pMem->flags & MEM_Null)
return SQLITE_OK;
- if ((pMem->flags & MEM_Blob) != 0 && aff == AFFINITY_REAL) {
+ if ((pMem->flags & MEM_Blob) != 0 && type == FIELD_TYPE_NUMBER) {
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i, pMem->n) == 0) {
MemSetTypeFlag(pMem, MEM_Real);
pMem->u.r = pMem->u.i;
@@ -610,8 +611,8 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
}
return ! sqlite3AtoF(pMem->z, &pMem->u.r, pMem->n);
}
- switch (aff) {
- case AFFINITY_BLOB:
+ switch (type) {
+ case FIELD_TYPE_SCALAR:
if (pMem->flags & MEM_Blob)
return SQLITE_OK;
if (pMem->flags & MEM_Str) {
@@ -625,7 +626,7 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
return 0;
}
return SQLITE_ERROR;
- case AFFINITY_INTEGER:
+ case FIELD_TYPE_INTEGER:
if ((pMem->flags & MEM_Blob) != 0) {
if (sql_atoi64(pMem->z, (int64_t *) &pMem->u.i,
pMem->n) != 0)
@@ -634,13 +635,13 @@ sqlite3VdbeMemCast(Mem * pMem, u8 aff)
return 0;
}
return sqlite3VdbeMemIntegerify(pMem, true);
- case AFFINITY_REAL:
+ case FIELD_TYPE_NUMBER:
return sqlite3VdbeMemRealify(pMem);
default:
- assert(aff == AFFINITY_TEXT);
+ assert(type == FIELD_TYPE_STRING);
assert(MEM_Str == (MEM_Blob >> 3));
pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
- sqlite3ValueApplyAffinity(pMem, AFFINITY_TEXT);
+ sqlite3ValueApplyAffinity(pMem, FIELD_TYPE_STRING);
assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
pMem->flags &= ~(MEM_Int | MEM_Real | MEM_Blob | MEM_Zero);
return SQLITE_OK;
@@ -1579,6 +1580,7 @@ sqlite3Stat4ProbeSetValue(Parse * pParse, /* Parse context */
u8 aff = sql_space_index_part_affinity(space->def, idx,
iVal + i);
alloc.iVal = iVal + i;
+ aff = sql_affinity_to_field_type(aff);
rc = stat4ValueFromExpr(pParse, pElem, aff, &alloc,
&pVal);
if (!pVal)
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 571b5af78..6b8999dd9 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
int nLower = -1;
int nUpper = index->def->opts.stat->sample_count + 1;
int rc = SQLITE_OK;
- u8 aff = sql_space_index_part_affinity(space->def, p, nEq);
+ enum field_type type = p->key_def->parts[nEq].type;
sqlite3_value *p1 = 0; /* Value extracted from pLower */
sqlite3_value *p2 = 0; /* Value extracted from pUpper */
@@ -1209,12 +1209,12 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
struct coll *coll = p->key_def->parts[nEq].coll;
if (pLower) {
rc = sqlite3Stat4ValueFromExpr(pParse, pLower->pExpr->pRight,
- aff, &p1);
+ type, &p1);
nLower = 0;
}
if (pUpper && rc == SQLITE_OK) {
rc = sqlite3Stat4ValueFromExpr(pParse, pUpper->pExpr->pRight,
- aff, &p2);
+ type, &p2);
nUpper = p2 ? 0 : index->def->opts.stat->sample_count;
}
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index b124a1d53..efbc91cf8 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -364,7 +364,7 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
}
/*
- * Code an OP_Affinity opcode to apply the column affinity string zAff
+ * Code an OP_ApplyType opcode to apply the column type string types
* to the n registers starting at base.
*
* As an optimization, AFFINITY_BLOB entries (which are no-ops) at the
@@ -396,9 +396,12 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
n--;
}
- /* Code the OP_Affinity opcode if there is anything left to do. */
if (n > 0) {
- sqlite3VdbeAddOp4(v, OP_Affinity, base, n, 0, zAff, n);
+ enum field_type *types =
+ sql_affinity_str_to_field_type_str(zAff);
+ types[n] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *)types,
+ P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(pParse, base, n);
}
}
diff --git a/test/sql-tap/cast.test.lua b/test/sql-tap/cast.test.lua
index 51d557937..189ca8933 100755
--- a/test/sql-tap/cast.test.lua
+++ b/test/sql-tap/cast.test.lua
@@ -70,7 +70,7 @@ test:do_catchsql_test(
SELECT CAST(x'616263' AS numeric)
]], {
-- <cast-1.5>
- 1, 'Type mismatch: can not convert abc to real'
+ 1, 'Type mismatch: can not convert abc to number'
-- </cast-1.5>
})
@@ -450,7 +450,7 @@ test:do_catchsql_test(
SELECT CAST('123abc' AS numeric)
]], {
-- <cast-1.45>
- 1, 'Type mismatch: can not convert 123abc to real'
+ 1, 'Type mismatch: can not convert 123abc to number'
-- </cast-1.45>
})
@@ -480,7 +480,7 @@ test:do_catchsql_test(
SELECT CAST('123.5abc' AS numeric)
]], {
-- <cast-1.51>
- 1, 'Type mismatch: can not convert 123.5abc to real'
+ 1, 'Type mismatch: can not convert 123.5abc to number'
-- </cast-1.51>
})
@@ -561,7 +561,7 @@ test:do_catchsql_test(
SELECT CAST('abc' AS REAL)
]], {
-- <case-1.66>
- 1, 'Type mismatch: can not convert abc to real'
+ 1, 'Type mismatch: can not convert abc to number'
-- </case-1.66>
})
@@ -875,7 +875,7 @@ test:do_test(
]]
end, {
-- <cast-4.4>
- 1, 'Type mismatch: can not convert abc to real'
+ 1, 'Type mismatch: can not convert abc to number'
-- </cast-4.4>
})
diff --git a/test/sql-tap/tkt-80e031a00f.test.lua b/test/sql-tap/tkt-80e031a00f.test.lua
index 2d4f81798..8517a581f 100755
--- a/test/sql-tap/tkt-80e031a00f.test.lua
+++ b/test/sql-tap/tkt-80e031a00f.test.lua
@@ -346,7 +346,7 @@ test:do_catchsql_test(
SELECT 'hello' IN t1
]], {
-- <tkt-80e031a00f.27>
- 1, 'Type mismatch: can not convert hello to real'
+ 1, 'Type mismatch: can not convert hello to number'
-- </tkt-80e031a00f.27>
})
@@ -356,7 +356,7 @@ test:do_catchsql_test(
SELECT 'hello' NOT IN t1
]], {
-- <tkt-80e031a00f.28>
- 1, 'Type mismatch: can not convert hello to real'
+ 1, 'Type mismatch: can not convert hello to number'
-- </tkt-80e031a00f.28>
})
@@ -386,7 +386,7 @@ test:do_catchsql_test(
SELECT x'303132' IN t1
]], {
-- <tkt-80e031a00f.31>
- 1, 'Type mismatch: can not convert 012 to real'
+ 1, 'Type mismatch: can not convert 012 to number'
-- </tkt-80e031a00f.31>
})
@@ -396,7 +396,7 @@ test:do_catchsql_test(
SELECT x'303132' NOT IN t1
]], {
-- <tkt-80e031a00f.32>
- 1, 'Type mismatch: can not convert 012 to real'
+ 1, 'Type mismatch: can not convert 012 to number'
-- </tkt-80e031a00f.32>
})
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2019-01-16 14:26 ` n.pettik
@ 2019-01-22 15:41 ` Vladislav Shpilevoy
2019-01-28 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-22 15:41 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
>>
>>> /* Set flag to save memory allocating one
>>> * by malloc.
>>> */
>>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>>> index 917e6e30b..c823c5a06 100644
>>> --- a/src/box/sql/expr.c
>>> +++ b/src/box/sql/expr.c
>>> @@ -2858,8 +2858,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>>> jmpIfDynamic = -1;
>>> }
>>> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
>>> + char type =
>>> + sql_affinity_to_field_type(affinity);
>>> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
>>> - 1, r2, &affinity, 1);
>>> + 1, r2, &type,
>>> + 1);
>>
>> 4. I do not understand. Is p4type of sqlite3VdbeAddOp4 of OP_MakeRecord
>> a length of an affinity string, or a type of its allocation? Or both?
>
> Both. p4type > 0 means that p4type bytes are copied to freshly allocated memory
> and P4 set to DYNAMIC. sqlite3DbStrNDup() in vdbeChangeP4Full() reserves
> one more byte for NULL termination.
I see now, that OP_MakeRecord uses only p4.z and does not touch p4type, so it makes
no sense to pass here length of the field_type array. Please, use only P4_STATIC and
P4_DYNAMIC. At this moment this opcode is mess: somewhere 0 is passed, somewhere
P4_DYNAMIC, in other places length of a field_type array.
>>> + switch (f_type) {
>>> + case FIELD_TYPE_INTEGER:
>>> + case FIELD_TYPE_UNSIGNED:
>>> if ((record->flags & MEM_Int) == MEM_Int)
>>> return 0;
>>> if ((record->flags & MEM_Real) == MEM_Real) {
>>> diff --git a/src/box/sql/where.c b/src/box/sql/where.c
>>> index 571b5af78..539296079 100644
>>> --- a/src/box/sql/where.c
>>> +++ b/src/box/sql/where.c
>>> @@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
>>> int nLower = -1;
>>> int nUpper = index->def->opts.stat->sample_count + 1;
>>> int rc = SQLITE_OK;
>>> - u8 aff = sql_space_index_part_affinity(space->def, p, nEq);
>>> + u8 aff = p->key_def->parts[nEq].type;
>>
>> 8. Why? Below in this function aff is used as affinity, not type.
>
> Am I missing smth?
> sqlite3Stat4ValueFromExpr -> stat4ValueFromExpr -> sqlite3ValueApplyAffinity -> mem_apply_type
Naming confuses. All this functions except mem_apply_type take "u8 affinity",
not "enum field_type type". Please, rename parameters and functions.
> Author: Nikita Pettik <korablev@tarantool.org>
> Date: Fri Dec 21 13:23:07 2018 +0200
>
> sql: replace affinity with field type for VDBE runtime
>
> This stage of affinity removal requires introducing of auxiliary
> intermediate function to convert array of affinity values to field type
> values. The rest of job done in this commit is a straightforward
> refactoring.
>
> Part of #3698
>
See below 6 comments.
> diff --git a/src/box/sql/build.c b/src/box/sql/build.c
> index e51e2db2a..514d0ca9d 100644
> --- a/src/box/sql/build.c
> +++ b/src/box/sql/build.c> diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
> index f9c42fdec..ca6c49373 100644
> --- a/src/box/sql/delete.c
> +++ b/src/box/sql/delete.c
> @@ -592,13 +592,13 @@ sql_generate_index_key(struct Parse *parse, struct index *index, int cursor,
> * is an integer, then it might be stored in the
> * table as an integer (using a compact
> * representation) then converted to REAL by an
> - * OP_RealAffinity opcode. But we are getting
> + * OP_Realify opcode. But we are getting
1. OP_RealAffinity still is mentioned in sqliteInt.h in
a comment.
> * ready to store this value back into an index,
> * where it should be converted by to INTEGER
> - * again. So omit the OP_RealAffinity opcode if
> + * again. So omit the OP_Realify opcode if
> * it is present
> */
> - sqlite3VdbeDeletePriorOpcode(v, OP_RealAffinity);
> + sqlite3VdbeDeletePriorOpcode(v, OP_Realify);
> }
> if (reg_out != 0)
> sqlite3VdbeAddOp3(v, OP_MakeRecord, reg_base, col_cnt, reg_out);
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index 0a80ca622..ca39faf51 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -3172,7 +3177,10 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
> * of the RHS using the LHS as a probe. If found, the result is
> * true.
> */
> - sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
> + enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
> + types[nVector] = field_type_MAX;
> + sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
> + P4_DYNAMIC);
2. Why do you need types[nVector] = field_type_MAX? As I see, before your patch
there was no additional zero termination.
> if (destIfFalse == destIfNull) {
> /* Combine Step 3 and Step 5 into a single opcode */
> sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,> diff --git a/src/box/sql/update.c b/src/box/sql/update.c
> index 0e2d0fde8..fd74817ea 100644
> --- a/src/box/sql/update.c
> +++ b/src/box/sql/update.c
> @@ -274,11 +274,12 @@ sqlite3Update(Parse * pParse, /* The parser context */
> nKey = pk_part_count;
> regKey = iPk;
> } else {
> - const char *zAff = is_view ? 0 :
> - sql_space_index_affinity_str(pParse->db, def,
> - pPk->def);
> + enum field_type *types = is_view ? NULL :
> + sql_index_type_str(pParse->db,
> + pPk->def);
> sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, pk_part_count,
> - regKey, zAff, pk_part_count);
> + regKey, (char *) types,
> + is_view ? 0 : P4_DYNAMIC);
3. I guess, here and in other places, where type str is either 0
or not 0, you can always set P4_DYNAMIC, it looks a bit simpler
and works as well. "is_view ? 0 : P4_DYNAMIC" -> "P4_DYNAMIC".
> /*
> * Set flag to save memory allocating one by
> * malloc.
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 4345af24e..61d73b676 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -306,32 +306,35 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
> }
>
> /**
> - * Processing is determine by the affinity parameter:
> + * Processing is determine by the field type parameter:
4. Not sure, if determine is an adjective or a noun. It is
always a verb. So it should be 'is determine*D*', shouldn't it?
> *
> - * AFFINITY_INTEGER:
> - * AFFINITY_REAL:
> - * Try to convert mem to an integer representation or a
> - * floating-point representation if an integer representation
> - * is not possible. Note that the integer representation is
> - * always preferred, even if the affinity is REAL, because
> - * an integer representation is more space efficient on disk.
> + * INTEGER:
> + * If memory holds floating point value and it can be
> + * converted without loss (2.0 - > 2), it's type is
> + * changed to INT. Otherwise, simply return success status.
> *
> - * AFFINITY_TEXT:
> - * Convert mem to a text representation.
> + * NUMBER:
> + * If memory holds INT or floating point value,
> + * no actions take place.
> *
> - * AFFINITY_BLOB:
> - * No-op. mem is unchanged.
> + * STRING:
> + * Convert mem to a string representation.
> *
> - * @param record The value to apply affinity to.
> - * @param affinity The affinity to be applied.
> + * SCALAR:
> + * Mem is unchanged, but flag is set to BLOB.
> + *
> + * @param record The value to apply type to.
> + * @param type_t The type to be applied.
5. type_t -> f_type. By the way, why f_type and not just type?
> */
> static int
> -mem_apply_affinity(struct Mem *record, enum affinity_type affinity)
> +mem_apply_type(struct Mem *record, enum field_type f_type)
> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
> index b124a1d53..efbc91cf8 100644
> --- a/src/box/sql/wherecode.c
> +++ b/src/box/sql/wherecode.c
> @@ -396,9 +396,12 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
> n--;
> }
>
> - /* Code the OP_Affinity opcode if there is anything left to do. */
> if (n > 0) {
> - sqlite3VdbeAddOp4(v, OP_Affinity, base, n, 0, zAff, n);
> + enum field_type *types =
> + sql_affinity_str_to_field_type_str(zAff);
> + types[n] = field_type_MAX;
6. The same as 2.
> + sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *)types,
> + P4_DYNAMIC);
> sqlite3ExprCacheAffinityChange(pParse, base, n);
> }
> }
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2019-01-22 15:41 ` Vladislav Shpilevoy
@ 2019-01-28 16:39 ` n.pettik
2019-01-30 13:04 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-28 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>>>> index 917e6e30b..c823c5a06 100644
>>>> --- a/src/box/sql/expr.c
>>>> +++ b/src/box/sql/expr.c
>>>> @@ -2858,8 +2858,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>>>> jmpIfDynamic = -1;
>>>> }
>>>> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
>>>> + char type =
>>>> + sql_affinity_to_field_type(affinity);
>>>> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
>>>> - 1, r2, &affinity, 1);
>>>> + 1, r2, &type,
>>>> + 1);
>>>
>>> 4. I do not understand. Is p4type of sqlite3VdbeAddOp4 of OP_MakeRecord
>>> a length of an affinity string, or a type of its allocation? Or both?
>> Both. p4type > 0 means that p4type bytes are copied to freshly allocated memory
>> and P4 set to DYNAMIC. sqlite3DbStrNDup() in vdbeChangeP4Full() reserves
>> one more byte for NULL termination.
>
> I see now, that OP_MakeRecord uses only p4.z and does not touch p4type, so it makes
> no sense to pass here length of the field_type array. Please, use only P4_STATIC and
> P4_DYNAMIC. At this moment this opcode is mess: somewhere 0 is passed, somewhere
> P4_DYNAMIC, in other places length of a field_type array.
Well, actually we can’t pass here neither of these args and that’s why:
STATIC and DYNAMIC indicate that memory which is passed through
P4 argument will be copied to new chunk of memory:
985 void
986 sqlite3VdbeChangeP4(Vdbe * p, int addr, const char *zP4, int n)
987 {
…
1008 if (n == P4_INT32) {
1009 /* Note: this cast is safe, because the origin data point was an int
1010 * that was cast to a (const char *).
1011 */
1012 pOp->p4.i = SQLITE_PTR_TO_INT(zP4);
1013 pOp->p4type = P4_INT32;
1014 } if (n == P4_BOOL) {
1015 pOp->p4.b = *(bool*)zP4;
1016 pOp->p4type = P4_BOOL;
1017 } else {
1018 assert(n < 0);
1019 pOp->p4.p = (void *)zP4;
1020 pOp->p4type = (signed char)n;
1021 }
1022 }
Both P4_STATIC and P4_DYNAMIC are less than 0, so pOp->p4.p points
to object on allocated on stack (in this particular case).
Passing n > 0 we are implying that n bytes of memory starting from zP4
must be copied to new chunk, which comes with P4_DYNAMIC
If you still have doubts, I assure you that I’ve already tried to replace it and
always get assertion faults like:
Assertion failed: (f_type < field_type_MAX), function mem_apply_type, file tarantool/src/box/sql/vdbe.c, line 334.
In other places (where we use P4_DYNAMIC), field_type array is allocated
on heap, so it can be freed after VDBE execution.
As an option we can allocate on heap this string before calling
sqlite3VdbeChangeP4, like this:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index ca39faf51..27bbc1b1c 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -2860,11 +2860,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
enum field_type type =
sql_affinity_to_field_type(affinity);
- enum field_type types[2] =
- { type, field_type_MAX };
+ size_t sz = 2 * sizeof(enum field_type);
+ enum field_type *types =
+ sqlite3DbMallocRaw(pParse->db,
+ sz);
+ types[0] = type;
+ types[1] = field_type_MAX;
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
1, r2, (char *)types,
- sizeof(types));
+ P4_DYNAMIC);
>>>> diff --git a/src/box/sql/where.c b/src/box/sql/where.c
>>>> index 571b5af78..539296079 100644
>>>> --- a/src/box/sql/where.c
>>>> +++ b/src/box/sql/where.c
>>>> @@ -1200,7 +1200,7 @@ whereRangeSkipScanEst(Parse * pParse, /* Parsing & code generating context */
>>>> int nLower = -1;
>>>> int nUpper = index->def->opts.stat->sample_count + 1;
>>>> int rc = SQLITE_OK;
>>>> - u8 aff = sql_space_index_part_affinity(space->def, p, nEq);
>>>> + u8 aff = p->key_def->parts[nEq].type;
>>>
>>> 8. Why? Below in this function aff is used as affinity, not type.
>> Am I missing smth?
>> sqlite3Stat4ValueFromExpr -> stat4ValueFromExpr -> sqlite3ValueApplyAffinity -> mem_apply_type
>
> Naming confuses. All this functions except mem_apply_type take "u8 affinity",
> not "enum field_type type". Please, rename parameters and functions.
As you wish. I just didn’t want to mix functional
changes and non-functional refactoring since it
complicates review process.
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 579f68fed..462cd233f 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -4375,8 +4375,9 @@ void sqlite3ValueSetStr(sqlite3_value *, int, const void *,
void sqlite3ValueSetNull(sqlite3_value *);
void sqlite3ValueFree(sqlite3_value *);
sqlite3_value *sqlite3ValueNew(sqlite3 *);
-int sqlite3ValueFromExpr(sqlite3 *, Expr *, u8, sqlite3_value **);
-void sqlite3ValueApplyAffinity(sqlite3_value *, u8);
+int sqlite3ValueFromExpr(sqlite3 *, Expr *, enum field_type type,
+ sqlite3_value **);
+void sqlite3ValueApplyAffinity(sqlite3_value *, enum field_type type);
#ifndef SQLITE_AMALGAMATION
extern const unsigned char sqlite3OpcodeProperty[];
extern const char sqlite3StrBINARY[];
@@ -4600,7 +4601,8 @@ int sqlite3ExprCheckIN(Parse *, Expr *);
void sqlite3AnalyzeFunctions(void);
int sqlite3Stat4ProbeSetValue(Parse *, struct index_def *, UnpackedRecord **, Expr *, int,
int, int *);
-int sqlite3Stat4ValueFromExpr(Parse *, Expr *, u8, sqlite3_value **);
+int sqlite3Stat4ValueFromExpr(Parse *, Expr *, enum field_type type,
+ sqlite3_value **);
void sqlite3Stat4ProbeFree(UnpackedRecord *);
/**
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 61d73b676..32be5b9fa 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -394,9 +394,9 @@ int sqlite3_value_numeric_type(sqlite3_value *pVal) {
void
sqlite3ValueApplyAffinity(
sqlite3_value *pVal,
- u8 affinity)
+ enum field_type type)
{
- mem_apply_type((Mem *) pVal, affinity);
+ mem_apply_type((Mem *) pVal, type);
}
/*
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index cd71641b0..7dc5dbfa9 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -1161,7 +1161,7 @@ valueNew(sqlite3 * db, struct ValueNewStat4Ctx *p)
static int
valueFromFunction(sqlite3 * db, /* The database connection */
Expr * p, /* The expression to evaluate */
- u8 aff, /* Affinity to use */
+ enum field_type type,
sqlite3_value ** ppVal, /* Write the new value here */
struct ValueNewStat4Ctx *pCtx /* Second argument for valueNew() */
)
@@ -1199,7 +1199,7 @@ valueFromFunction(sqlite3 * db, /* The database connection */
}
for (i = 0; i < nVal; i++) {
rc = sqlite3ValueFromExpr(db, pList->a[i].pExpr,
- aff, &apVal[i]);
+ type, &apVal[i]);
if (apVal[i] == 0 || rc != SQLITE_OK)
goto value_from_function_out;
}
@@ -1220,7 +1220,7 @@ valueFromFunction(sqlite3 * db, /* The database connection */
rc = ctx.isError;
sqlite3ErrorMsg(pCtx->pParse, "%s", sqlite3_value_text(pVal));
} else {
- sqlite3ValueApplyAffinity(pVal, aff);
+ sqlite3ValueApplyAffinity(pVal, type);
assert(rc == SQLITE_OK);
}
pCtx->pParse->rc = rc;
@@ -1253,7 +1253,7 @@ valueFromFunction(sqlite3 * db, /* The database connection */
static int
valueFromExpr(sqlite3 * db, /* The database connection */
Expr * pExpr, /* The expression to evaluate */
- u8 affinity, /* Affinity to use */
+ enum field_type type,
sqlite3_value ** ppVal, /* Write the new value here */
struct ValueNewStat4Ctx *pCtx /* Second argument for valueNew() */
)
@@ -1284,7 +1284,7 @@ valueFromExpr(sqlite3 * db, /* The database connection */
testcase(rc != SQLITE_OK);
if (*ppVal) {
sqlite3VdbeMemCast(*ppVal, aff);
- sqlite3ValueApplyAffinity(*ppVal, affinity);
+ sqlite3ValueApplyAffinity(*ppVal, type);
}
return rc;
}
@@ -1315,18 +1315,18 @@ valueFromExpr(sqlite3 * db, /* The database connection */
goto no_mem;
sqlite3ValueSetStr(pVal, -1, zVal, SQLITE_DYNAMIC);
}
- if ((op == TK_INTEGER || op == TK_FLOAT)
- && affinity == AFFINITY_BLOB) {
- sqlite3ValueApplyAffinity(pVal, AFFINITY_REAL);
+ if ((op == TK_INTEGER || op == TK_FLOAT) &&
+ type == FIELD_TYPE_SCALAR) {
+ sqlite3ValueApplyAffinity(pVal, FIELD_TYPE_NUMBER);
} else {
- sqlite3ValueApplyAffinity(pVal, affinity);
+ sqlite3ValueApplyAffinity(pVal, type);
}
if (pVal->flags & (MEM_Int | MEM_Real))
pVal->flags &= ~MEM_Str;
} else if (op == TK_UMINUS) {
/* This branch happens for multiple negative signs. Ex: -(-5) */
if (SQLITE_OK ==
- sqlite3ValueFromExpr(db, pExpr->pLeft, affinity, &pVal)
+ sqlite3ValueFromExpr(db, pExpr->pLeft, type, &pVal)
&& pVal != 0) {
if ((rc = sqlite3VdbeMemNumerify(pVal)) != SQLITE_OK)
return rc;
@@ -1338,7 +1338,7 @@ valueFromExpr(sqlite3 * db, /* The database connection */
} else {
pVal->u.i = -pVal->u.i;
}
- sqlite3ValueApplyAffinity(pVal, affinity);
+ sqlite3ValueApplyAffinity(pVal, type);
}
} else if (op == TK_NULL) {
pVal = valueNew(db, pCtx);
@@ -1364,7 +1364,7 @@ valueFromExpr(sqlite3 * db, /* The database connection */
#endif
else if (op == TK_FUNCTION && pCtx != 0) {
- rc = valueFromFunction(db, pExpr, affinity, &pVal, pCtx);
+ rc = valueFromFunction(db, pExpr, type, &pVal, pCtx);
}
*ppVal = pVal;
@@ -1393,11 +1393,11 @@ valueFromExpr(sqlite3 * db, /* The database connection */
int
sqlite3ValueFromExpr(sqlite3 * db, /* The database connection */
Expr * pExpr, /* The expression to evaluate */
- u8 affinity, /* Affinity to use */
+ enum field_type type,
sqlite3_value ** ppVal /* Write the new value here */
)
{
- return pExpr ? valueFromExpr(db, pExpr, affinity, ppVal, 0) : 0;
+ return pExpr ? valueFromExpr(db, pExpr, type, ppVal, 0) : 0;
}
/*
@@ -1471,7 +1471,7 @@ sqlite3AnalyzeFunctions(void)
static int
stat4ValueFromExpr(Parse * pParse, /* Parse context */
Expr * pExpr, /* The expression to extract a value from */
- u8 affinity, /* Affinity to use */
+ enum field_type type,
struct ValueNewStat4Ctx *pAlloc, /* How to allocate space. Or NULL */
sqlite3_value ** ppVal /* OUT: New value object (or NULL) */
)
@@ -1501,14 +1501,13 @@ stat4ValueFromExpr(Parse * pParse, /* Parse context */
rc = sqlite3VdbeMemCopy((Mem *) pVal,
&v->aVar[iBindVar - 1]);
if (rc == SQLITE_OK) {
- sqlite3ValueApplyAffinity(pVal,
- affinity);
+ sqlite3ValueApplyAffinity(pVal, type);
}
pVal->db = pParse->db;
}
}
} else {
- rc = valueFromExpr(db, pExpr, affinity, &pVal, pAlloc);
+ rc = valueFromExpr(db, pExpr, type, &pVal, pAlloc);
}
assert(pVal == 0 || pVal->db == db);
@@ -1606,11 +1605,11 @@ sqlite3Stat4ProbeSetValue(Parse * pParse, /* Parse context */
int
sqlite3Stat4ValueFromExpr(Parse * pParse, /* Parse context */
Expr * pExpr, /* The expression to extract a value from */
- u8 affinity, /* Affinity to use */
+ enum field_type type,
sqlite3_value ** ppVal /* OUT: New value object (or NULL) */
)
{
- return stat4ValueFromExpr(pParse, pExpr, affinity, 0, ppVal);
+ return stat4ValueFromExpr(pParse, pExpr, type, 0, ppVal);
>> diff --git a/src/box/sql/build.c b/src/box/sql/build.c
>> index e51e2db2a..514d0ca9d 100644
>> --- a/src/box/sql/build.c
>> +++ b/src/box/sql/build.c> diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
>> index f9c42fdec..ca6c49373 100644
>> --- a/src/box/sql/delete.c
>> +++ b/src/box/sql/delete.c
>> @@ -592,13 +592,13 @@ sql_generate_index_key(struct Parse *parse, struct index *index, int cursor,
>> * is an integer, then it might be stored in the
>> * table as an integer (using a compact
>> * representation) then converted to REAL by an
>> - * OP_RealAffinity opcode. But we are getting
>> + * OP_Realify opcode. But we are getting
>
> 1. OP_RealAffinity still is mentioned in sqliteInt.h in
> a comment.
Removed:
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 579f68fed..cbdd1f1a3 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -4459,7 +4460,7 @@ int sqlite3ResolveOrderGroupBy(Parse *, Select *, ExprList *, const char *);
* function is capable of transforming these types of expressions into
* sqlite3_value objects.
*
- * If parameter iReg is not negative, code an OP_RealAffinity instruction
+ * If parameter iReg is not negative, code an OP_Realify instruction
* on register iReg. This is used when an equivalent integer value is
* stored in place of an 8-byte floating point value in order to save
* space.
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index 0a80ca622..ca39faf51 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -3172,7 +3177,10 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
>> * of the RHS using the LHS as a probe. If found, the result is
>> * true.
>> */
>> - sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
>> + enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
>> + types[nVector] = field_type_MAX;
>> + sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
>> + P4_DYNAMIC);
>
> 2. Why do you need types[nVector] = field_type_MAX? As I see, before your patch
> there was no additional zero termination.
To cut off array of types. Before my patch, first nVector bytes
were copied inside vdbeChangeP4Full() and automatically
terminated with NULL (sqlite3DbStrNDup).
>> sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
Now it makes no sense to copy this string: we can set
array termination and pass it with dynamic flag.
>> --- a/src/box/sql/update.c
>> +++ b/src/box/sql/update.c
>> @@ -274,11 +274,12 @@ sqlite3Update(Parse * pParse, /* The parser context */
>> nKey = pk_part_count;
>> regKey = iPk;
>> } else {
>> - const char *zAff = is_view ? 0 :
>> - sql_space_index_affinity_str(pParse->db, def,
>> - pPk->def);
>> + enum field_type *types = is_view ? NULL :
>> + sql_index_type_str(pParse->db,
>> + pPk->def);
>> sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, pk_part_count,
>> - regKey, zAff, pk_part_count);
>> + regKey, (char *) types,
>> + is_view ? 0 : P4_DYNAMIC);
>
> 3. I guess, here and in other places, where type str is either 0
> or not 0, you can always set P4_DYNAMIC, it looks a bit simpler
> and works as well. "is_view ? 0 : P4_DYNAMIC" -> "P4_DYNAMIC”.
Ok, indeed it is valid replacement:
diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
index ca6c49373..a6e7547f5 100644
--- a/src/box/sql/delete.c
+++ b/src/box/sql/delete.c
@@ -336,8 +336,7 @@ sql_table_delete_from(struct Parse *parse, struct SrcList *tab_list,
sql_index_type_str(parse->db,
pk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, reg_pk, pk_len,
- reg_key, (char *)types, is_view ? 0 :
- P4_DYNAMIC);
+ reg_key, (char *)types, P4_DYNAMIC);
/* Set flag to save memory allocating one
* by malloc.
diff --git a/src/box/sql/update.c b/src/box/sql/update.c
index fd74817ea..cea840a99 100644
--- a/src/box/sql/update.c
+++ b/src/box/sql/update.c
@@ -278,8 +278,7 @@ sqlite3Update(Parse * pParse, /* The parser context */
sql_index_type_str(pParse->db,
pPk->def);
sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, pk_part_count,
- regKey, (char *) types,
- is_view ? 0 : P4_DYNAMIC);
+ regKey, (char *) types, P4_DYNAMIC);
/*
* Set flag to save memory allocating one by
* malloc.
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 4345af24e..61d73b676 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -306,32 +306,35 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
>> }
>> /**
>> - * Processing is determine by the affinity parameter:
>> + * Processing is determine by the field type parameter:
>
> 4. Not sure, if determine is an adjective or a noun. It is
> always a verb. So it should be 'is determine*D*', shouldn't it?
Yep, it is my (or not my) mistake. Fixed:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 61d73b676..a38075e2e 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -306,7 +306,7 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
}
/**
- * Processing is determine by the field type parameter:
+ * Processing is determined by the field type parameter:
*
>> *
>> - * AFFINITY_INTEGER:
>> - * AFFINITY_REAL:
>> - * Try to convert mem to an integer representation or a
>> - * floating-point representation if an integer representation
>> - * is not possible. Note that the integer representation is
>> - * always preferred, even if the affinity is REAL, because
>> - * an integer representation is more space efficient on disk.
>> + * INTEGER:
>> + * If memory holds floating point value and it can be
>> + * converted without loss (2.0 - > 2), it's type is
>> + * changed to INT. Otherwise, simply return success status.
>> *
>> - * AFFINITY_TEXT:
>> - * Convert mem to a text representation.
>> + * NUMBER:
>> + * If memory holds INT or floating point value,
>> + * no actions take place.
>> *
>> - * AFFINITY_BLOB:
>> - * No-op. mem is unchanged.
>> + * STRING:
>> + * Convert mem to a string representation.
>> *
>> - * @param record The value to apply affinity to.
>> - * @param affinity The affinity to be applied.
>> + * SCALAR:
>> + * Mem is unchanged, but flag is set to BLOB.
>> + *
>> + * @param record The value to apply type to.
>> + * @param type_t The type to be applied.
>
> 5. type_t -> f_type. By the way, why f_type and not just type?
Hmm, I was sure that I got compile error since other
variable already occupied this name…
Nevermind, lets use simple ’type’ name:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 61d73b676..17921ce36 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -324,15 +324,15 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
* Mem is unchanged, but flag is set to BLOB.
*
* @param record The value to apply type to.
- * @param type_t The type to be applied.
+ * @param type The type to be applied.
*/
static int
-mem_apply_type(struct Mem *record, enum field_type f_type)
+mem_apply_type(struct Mem *record, enum field_type type)
{
if ((record->flags & MEM_Null) != 0)
return 0;
- assert(f_type < field_type_MAX);
- switch (f_type) {
+ assert(type < field_type_MAX);
+ switch (type) {
case FIELD_TYPE_INTEGER:
case FIELD_TYPE_UNSIGNED:
if ((record->flags & MEM_Int) == MEM_Int)
>> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
>> index b124a1d53..efbc91cf8 100644
>> --- a/src/box/sql/wherecode.c
>> +++ b/src/box/sql/wherecode.c
>> @@ -396,9 +396,12 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
>> n--;
>> }
>> - /* Code the OP_Affinity opcode if there is anything left to do. */
>> if (n > 0) {
>> - sqlite3VdbeAddOp4(v, OP_Affinity, base, n, 0, zAff, n);
>> + enum field_type *types =
>> + sql_affinity_str_to_field_type_str(zAff);
>> + types[n] = field_type_MAX;
>
> 6. The same as 2.
See answer above: here we need only first n fields,
not the whole array.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2019-01-28 16:39 ` n.pettik
@ 2019-01-30 13:04 ` Vladislav Shpilevoy
2019-02-01 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-30 13:04 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
On 28/01/2019 19:39, n.pettik wrote:
>
>>>>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>>>>> index 917e6e30b..c823c5a06 100644
>>>>> --- a/src/box/sql/expr.c
>>>>> +++ b/src/box/sql/expr.c
>>>>> @@ -2858,8 +2858,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>>>>> jmpIfDynamic = -1;
>>>>> }
>>>>> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
>>>>> + char type =
>>>>> + sql_affinity_to_field_type(affinity);
>>>>> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
>>>>> - 1, r2, &affinity, 1);
>>>>> + 1, r2, &type,
>>>>> + 1);
>>>>
>>>> 4. I do not understand. Is p4type of sqlite3VdbeAddOp4 of OP_MakeRecord
>>>> a length of an affinity string, or a type of its allocation? Or both?
>>> Both. p4type > 0 means that p4type bytes are copied to freshly allocated memory
>>> and P4 set to DYNAMIC. sqlite3DbStrNDup() in vdbeChangeP4Full() reserves
>>> one more byte for NULL termination.
>>
>> I see now, that OP_MakeRecord uses only p4.z and does not touch p4type, so it makes
>> no sense to pass here length of the field_type array. Please, use only P4_STATIC and
>> P4_DYNAMIC. At this moment this opcode is mess: somewhere 0 is passed, somewhere
>> P4_DYNAMIC, in other places length of a field_type array.
>
> Well, actually we can’t pass here neither of these args and that’s why:
> STATIC and DYNAMIC indicate that memory which is passed through
> P4 argument will be copied to new chunk of memory:
>
> 985 void
> 986 sqlite3VdbeChangeP4(Vdbe * p, int addr, const char *zP4, int n)
> 987 {
>
> …
>
> 1008 if (n == P4_INT32) {
> 1009 /* Note: this cast is safe, because the origin data point was an int
> 1010 * that was cast to a (const char *).
> 1011 */
> 1012 pOp->p4.i = SQLITE_PTR_TO_INT(zP4);
> 1013 pOp->p4type = P4_INT32;
> 1014 } if (n == P4_BOOL) {
> 1015 pOp->p4.b = *(bool*)zP4;
> 1016 pOp->p4type = P4_BOOL;
> 1017 } else {
> 1018 assert(n < 0);
> 1019 pOp->p4.p = (void *)zP4;
> 1020 pOp->p4type = (signed char)n;
> 1021 }
> 1022 }
>
> Both P4_STATIC and P4_DYNAMIC are less than 0, so pOp->p4.p points
> to object on allocated on stack (in this particular case).
>
> Passing n > 0 we are implying that n bytes of memory starting from zP4
> must be copied to new chunk, which comes with P4_DYNAMIC
>
> If you still have doubts, I assure you that I’ve already tried to replace it and
> always get assertion faults like:
> Assertion failed: (f_type < field_type_MAX), function mem_apply_type, file tarantool/src/box/sql/vdbe.c, line 334.
>
> In other places (where we use P4_DYNAMIC), field_type array is allocated
> on heap, so it can be freed after VDBE execution.
Ok, now I got it. Thanks for the explanation.
>
> As an option we can allocate on heap this string before calling
> sqlite3VdbeChangeP4, like this:
>
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index ca39faf51..27bbc1b1c 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -2860,11 +2860,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
> enum field_type type =
> sql_affinity_to_field_type(affinity);
> - enum field_type types[2] =
> - { type, field_type_MAX };
> + size_t sz = 2 * sizeof(enum field_type);
> + enum field_type *types =
> + sqlite3DbMallocRaw(pParse->db,
> + sz);
> + types[0] = type;
> + types[1] = field_type_MAX;
> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
> 1, r2, (char *)types,
> - sizeof(types));
> + P4_DYNAMIC);
No, never mind. The current way is ok.
>>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>>> index 0a80ca622..ca39faf51 100644
>>> --- a/src/box/sql/expr.c
>>> +++ b/src/box/sql/expr.c
>>> @@ -3172,7 +3177,10 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
>>> * of the RHS using the LHS as a probe. If found, the result is
>>> * true.
>>> */
>>> - sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
>>> + enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
>>> + types[nVector] = field_type_MAX;
>>> + sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
>>> + P4_DYNAMIC);
>>
>> 2. Why do you need types[nVector] = field_type_MAX? As I see, before your patch
>> there was no additional zero termination.
>
> To cut off array of types. Before my patch, first nVector bytes
> were copied inside vdbeChangeP4Full() and automatically
> terminated with NULL (sqlite3DbStrNDup).
Still do not understand. As I see, zAff was taken from exprINAffinity().
This function allocates a string of affinities of length
sqlite3ExprVectorSize(pExpr->pLeft). But nVector was initialized with
exactly the same value. So strlen(zAff) should be equal to nVector, and
your function sql_affinity_str_to_field_type_str() should return already
correct array. I tried to test it and faced into a very strange situation.
Firstly, it appeared that already at the beginning of the
function, after this code:
zAff = exprINAffinity(pParse, pExpr);
nVector = sqlite3ExprVectorSize(pExpr->pLeft);
They are not already equal. This assertion fails:
strlen(zAff) == nVector.
I tried to print their values in debug, and found, that
zAff is stored in a buffer of length 2, but both bytes are
zeros. Why? It is a first strange situation.
Secondly, your function sql_affinity_str_to_field_type_str()
in such a case returns a buffer of one byte - [field_type_MAX],
because it uses strlen(affinity_str) + 1, which == 1 here.
Further, you do types[nVector] = field_type_MAX, so you are
out of array bounds, because nVector == 1, not 0. The only
reason why it does not fail is that malloc returns more
memory than necessary.
Please, check out if I am right, and fix the bug if so.
The assertion fails in sql-tap/randexpr1.test.lua.
Also, see 2 comments below.
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index 58e35e0d9..d263c1bb5 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -2854,8 +2854,17 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
> jmpIfDynamic = -1;
> }
> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
> + enum field_type type =
> + sql_affinity_to_field_type(affinity);
> + size_t sz = 2 * sizeof(enum field_type);
> + enum field_type *types =
> + sqlite3DbMallocRaw(pParse->db,
> + sz);
> + types[0] = type;
> + types[1] = field_type_MAX;
1. Please, either check for malloc result, or just use the
same method as in other places - declare it on the stack and pass
sizeof(this_array) instead of P4_DYNAMIC.
> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
> - 1, r2, &affinity, 1);
> + 1, r2, (char *)types,
> + P4_DYNAMIC);
> sqlite3ExprCacheAffinityChange(pParse,
> r3, 1);
> sqlite3VdbeAddOp2(v, OP_IdxInsert, r2,
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 24d992284..80d2fd0aa 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -2881,7 +2873,7 @@ case OP_MakeRecord: {
> * of the record to data0.
> */
> nField = pOp->p1;
> - zAffinity = pOp->p4.z;
> + enum field_type *types = (enum field_type *)pOp->p4.z;
2. Maybe, it is worth adding enum field_type *types into VdbeOp.p4union
and do not cast. Like we did with many other pointers.
> bIsEphemeral = pOp->p5;
> assert(nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem+1 - p->nCursor)+1);
> pData0 = &aMem[nField];
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2019-01-30 13:04 ` Vladislav Shpilevoy
@ 2019-02-01 16:39 ` n.pettik
2019-02-05 15:08 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-02-01 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>>>> index 0a80ca622..ca39faf51 100644
>>>> --- a/src/box/sql/expr.c
>>>> +++ b/src/box/sql/expr.c
>>>> @@ -3172,7 +3177,10 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
>>>> * of the RHS using the LHS as a probe. If found, the result is
>>>> * true.
>>>> */
>>>> - sqlite3VdbeAddOp4(v, OP_Affinity, rLhs, nVector, 0, zAff, nVector);
>>>> + enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
>>>> + types[nVector] = field_type_MAX;
>>>> + sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
>>>> + P4_DYNAMIC);
>>>
>>> 2. Why do you need types[nVector] = field_type_MAX? As I see, before your patch
>>> there was no additional zero termination.
>> To cut off array of types. Before my patch, first nVector bytes
>> were copied inside vdbeChangeP4Full() and automatically
>> terminated with NULL (sqlite3DbStrNDup).
>
> Still do not understand. As I see, zAff was taken from exprINAffinity().
> This function allocates a string of affinities of length
> sqlite3ExprVectorSize(pExpr->pLeft). But nVector was initialized with
> exactly the same value. So strlen(zAff) should be equal to nVector, and
> your function sql_affinity_str_to_field_type_str() should return already
> correct array. I tried to test it and faced into a very strange situation.
>
> Firstly, it appeared that already at the beginning of the
> function, after this code:
>
> zAff = exprINAffinity(pParse, pExpr);
> nVector = sqlite3ExprVectorSize(pExpr->pLeft);
>
> They are not already equal. This assertion fails:
>
> strlen(zAff) == nVector.
>
> I tried to print their values in debug, and found, that
> zAff is stored in a buffer of length 2, but both bytes are
> zeros. Why? It is a first strange situation.
In a nutshell: owing to the fact that AFFINITY_UNDEFINED equals to 0.
Long story:
Consider simple example:
SELECT NULL IN (2, 3, 4, NULL)
When we reach sqlite3ExprCodeIN(), pExpr represents AST with the root in IN operator:
(lldb) print (int)pExpr->op
(int) $4 = 12
#define TK_IN 12
LHS of the root is NULL (terminal symbol):
(lldb) print *pExpr->pLeft
(Expr) $6 = {
op = 'K'
= (affinity = AFFINITY_UNDEFINED, on_conflict_action = ON_CONFLICT_ACTION_NONE)
flags = 8388612
u = (zToken = "NULL", iValue = 59018336)
So nVector is 1. Moreover, NULL's affinity is UNDEFINED.
That is due to assigment to all expression by default
UNDEFINED affinity (as you may remember, I fixed it in
the next patch by setting it to SCALAR):
811 /* Construct a new Expr object from a single identifier. Use the
812 ** new Expr to populate pOut. Set the span of pOut to be the identifier
813 ** that created the expression.
814 */
815 static void spanExpr(ExprSpan *pOut, Parse *pParse, int op, Token t){
816 Expr *p = sqlite3DbMallocRawNN(pParse->db, sizeof(Expr)+t.n+1);
817 if( p ){
818 memset(p, 0, sizeof(Expr));
819 switch (op) {
820 case TK_STRING:
821 p->affinity = AFFINITY_TEXT;
822 break;
823 case TK_BLOB:
824 p->affinity = AFFINITY_BLOB;
Since NULL token is out of TK_STRING, TK_BLOB etc enumeration,
its affinity is 0, i.e. UNDERFINED.
Therefore, we see that nVector is 1, so zAff is of lenght 2
(because of additional null-termination), and finally,
sql_affinity_str_to_field_type_str() returns shorter array,
then is should be.
Fortunately, it doesn’t affect further patches, since there
we get rid of affinity, and SCALAR in enum is not 0.
> Secondly, your function sql_affinity_str_to_field_type_str()
> in such a case returns a buffer of one byte - [field_type_MAX],
> because it uses strlen(affinity_str) + 1, which == 1 here.
>
> Further, you do types[nVector] = field_type_MAX, so you are
> out of array bounds, because nVector == 1, not 0. The only
> reason why it does not fail is that malloc returns more
> memory than necessary.
>
> Please, check out if I am right, and fix the bug if so.
>
> The assertion fails in sql-tap/randexpr1.test.lua.
As a workaround (this problem is automatically resolved
in next patch) I suggest to pass to sql_affinity_str_to..
len of affinity array:
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 514d0ca9d..3adc6e262 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -521,17 +521,16 @@ sql_field_type_to_affinity(enum field_type field_type)
}
enum field_type *
-sql_affinity_str_to_field_type_str(const char *affinity_str)
+sql_affinity_str_to_field_type_str(const char *affinity_str, uint32_t len)
{
if (affinity_str == NULL)
return NULL;
- size_t len = strlen(affinity_str) + 1;
- size_t sz = len * sizeof(enum field_type);
+ size_t sz = (len + 1) * sizeof(enum field_type);
enum field_type *types =
(enum field_type *) sqlite3DbMallocRaw(sql_get(), sz);
- for (uint32_t i = 0; i < len - 1; ++i)
+ for (uint32_t i = 0; i < len; ++i)
types[i] = sql_affinity_to_field_type(affinity_str[i]);
- types[len - 1] = field_type_MAX;
+ types[len] = field_type_MAX;
return types;
}
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index d263c1bb5..b02089bc2 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3177,8 +3177,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
- types[nVector] = field_type_MAX;
+ enum field_type *types = sql_affinity_str_to_field_type_str(zAff,
+ nVector);
sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
P4_DYNAMIC);
if (destIfFalse == destIfNull) {
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 4b6983842..ef4514374 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1204,7 +1204,8 @@ selectInnerLoop(Parse * pParse, /* The parser context */
(unsigned int)nResultCol);
enum field_type *types =
sql_affinity_str_to_field_type_str(
- pDest->zAffSdst);
+ pDest->zAffSdst,
+ strlen(pDest->zAffSdst));
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
nResultCol, r1, (char *)types,
P4_DYNAMIC);
@@ -1631,7 +1632,8 @@ generateSortTail(Parse * pParse, /* Parsing context */
sqlite3Strlen30(pDest->zAffSdst));
enum field_type *types =
- sql_affinity_str_to_field_type_str(pDest->zAffSdst);
+ sql_affinity_str_to_field_type_str(pDest->zAffSdst,
+ strlen(pDest->zAffSdst));
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
regTupleid, (char *)types,
P4_DYNAMIC);
@@ -3070,7 +3072,8 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
enum field_type *types =
- sql_affinity_str_to_field_type_str(dest->zAffSdst);
+ sql_affinity_str_to_field_type_str(dest->zAffSdst,
+ strlen(dest->zAffSdst));
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
in->nSdst, r1, (char *)types,
P4_DYNAMIC);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 729e5257c..79999509c 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3454,7 +3454,7 @@ enum affinity_type
sql_field_type_to_affinity(enum field_type field_type);
enum field_type *
-sql_affinity_str_to_field_type_str(const char *affinity_str);
+sql_affinity_str_to_field_type_str(const char *affinity_str, uint32_t len);
/**
* Compile view, i.e. create struct Select from
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index efbc91cf8..e3e7c5222 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -398,8 +398,7 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
if (n > 0) {
enum field_type *types =
- sql_affinity_str_to_field_type_str(zAff);
- types[n] = field_type_MAX;
+ sql_affinity_str_to_field_type_str(zAff, n);
sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *)types,
P4_DYNAMIC);
> Also, see 2 comments below.
>
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index 58e35e0d9..d263c1bb5 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -2854,8 +2854,17 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>> jmpIfDynamic = -1;
>> }
>> r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
>> + enum field_type type =
>> + sql_affinity_to_field_type(affinity);
>> + size_t sz = 2 * sizeof(enum field_type);
>> + enum field_type *types =
>> + sqlite3DbMallocRaw(pParse->db,
>> + sz);
>> + types[0] = type;
>> + types[1] = field_type_MAX;
>
> 1. Please, either check for malloc result, or just use the
> same method as in other places - declare it on the stack and pass
> sizeof(this_array) instead of P4_DYNAMIC.
Sorry, I accidentally added it to diff - it was proposal to refactoring
(unsuccessful). Returned to its previous state.
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index d263c1bb5..5676d0720 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -2856,15 +2856,11 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
enum field_type type =
sql_affinity_to_field_type(affinity);
- size_t sz = 2 * sizeof(enum field_type);
- enum field_type *types =
- sqlite3DbMallocRaw(pParse->db,
- sz);
- types[0] = type;
- types[1] = field_type_MAX;
+ enum field_type types[2] =
+ { type, field_type_MAX };
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
1, r2, (char *)types,
- P4_DYNAMIC);
+ sizeof(types));
>> sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
>> - 1, r2, &affinity, 1);
>> + 1, r2, (char *)types,
>> + P4_DYNAMIC);
>> sqlite3ExprCacheAffinityChange(pParse,
>> r3, 1);
>> sqlite3VdbeAddOp2(v, OP_IdxInsert, r2,
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 24d992284..80d2fd0aa 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -2881,7 +2873,7 @@ case OP_MakeRecord: {
>> * of the record to data0.
>> */
>> nField = pOp->p1;
>> - zAffinity = pOp->p4.z;
>> + enum field_type *types = (enum field_type *)pOp->p4.z;
>
> 2. Maybe, it is worth adding enum field_type *types into VdbeOp.p4union
> and do not cast. Like we did with many other pointers.
Thanks for suggestion, but I guess it is not necessary now.
Lets consider this refactoring a bit later.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2019-02-01 16:39 ` n.pettik
@ 2019-02-05 15:08 ` Vladislav Shpilevoy
2019-02-05 17:46 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-02-05 15:08 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Hi! Thanks for the fixes!
>>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>>> index 24d992284..80d2fd0aa 100644
>>> --- a/src/box/sql/vdbe.c
>>> +++ b/src/box/sql/vdbe.c
>>> @@ -2881,7 +2873,7 @@ case OP_MakeRecord: {
>>> * of the record to data0.
>>> */
>>> nField = pOp->p1;
>>> - zAffinity = pOp->p4.z;
>>> + enum field_type *types = (enum field_type *)pOp->p4.z;
>>
>> 2. Maybe, it is worth adding enum field_type *types into VdbeOp.p4union
>> and do not cast. Like we did with many other pointers.
>
> Thanks for suggestion, but I guess it is not necessary now.
> Lets consider this refactoring a bit later.
>
Space * and other members of p4union also were not
necessary, but they make it easier to understand what can
be stored in p4, when you look at struct VdbeOp. So please,
apply this:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 80d2fd0aa..5e685389b 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2814,7 +2814,7 @@ case OP_Column: {
* memory cell in the range.
*/
case OP_ApplyType: {
- enum field_type *type_str = (enum field_type *)pOp->p4.z;
+ enum field_type *type_str = pOp->p4.types;
assert(type_str != NULL);
assert(type_str[pOp->p2] == field_type_MAX);
pIn1 = &aMem[pOp->p1];
@@ -2873,7 +2873,7 @@ case OP_MakeRecord: {
* of the record to data0.
*/
nField = pOp->p1;
- enum field_type *types = (enum field_type *)pOp->p4.z;
+ enum field_type *types = pOp->p4.types;
bIsEphemeral = pOp->p5;
assert(nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem+1 - p->nCursor)+1);
pData0 = &aMem[nField];
diff --git a/src/box/sql/vdbe.h b/src/box/sql/vdbe.h
index dcf974c51..088f24d0e 100644
--- a/src/box/sql/vdbe.h
+++ b/src/box/sql/vdbe.h
@@ -84,6 +84,11 @@ struct VdbeOp {
struct sql_key_info *key_info;
/** Used when p4type is P4_SPACEPTR. */
struct space *space;
+ /**
+ * Used to apply types when making a record, or
+ * doing a cast.
+ */
+ enum field_type *types;
} p4;
#ifdef SQLITE_ENABLE_EXPLAIN_COMMENTS
char *zComment; /* Comment to improve readability */
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 5/8] sql: replace field type with affinity for VDBE runtime
2019-02-05 15:08 ` Vladislav Shpilevoy
@ 2019-02-05 17:46 ` n.pettik
0 siblings, 0 replies; 48+ messages in thread
From: n.pettik @ 2019-02-05 17:46 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]
> On 5 Feb 2019, at 18:08, Vladislav Shpilevoy <v.shpilevoy@tarantool.org> wrote:
>
> Hi! Thanks for the fixes!
>
>>>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>>>> index 24d992284..80d2fd0aa 100644
>>>> --- a/src/box/sql/vdbe.c
>>>> +++ b/src/box/sql/vdbe.c
>>>> @@ -2881,7 +2873,7 @@ case OP_MakeRecord: {
>>>> * of the record to data0.
>>>> */
>>>> nField = pOp->p1;
>>>> - zAffinity = pOp->p4.z;
>>>> + enum field_type *types = (enum field_type *)pOp->p4.z;
>>>
>>> 2. Maybe, it is worth adding enum field_type *types into VdbeOp.p4union
>>> and do not cast. Like we did with many other pointers.
>> Thanks for suggestion, but I guess it is not necessary now.
>> Lets consider this refactoring a bit later.
>
> Space * and other members of p4union also were not
> necessary, but they make it easier to understand what can
> be stored in p4, when you look at struct VdbeOp. So please,
> apply this:
Ok, sorry, I slightly misunderstood your proposal:
I forgot that they would have the same layout and
we wouldn’t have to refactor sqlite3VdbeChangeP4()
with P4_FIELDTYPE and so on. Thanks, applied.
[-- Attachment #2: Type: text/html, Size: 6877 bytes --]
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] [PATCH 6/8] sql: replace affinity with field type in struct Expr
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (4 preceding siblings ...)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 5/8] sql: replace field type with affinity for VDBE runtime Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
2018-12-28 9:34 ` [tarantool-patches] [PATCH 7/8] sql: clean-up affinity from SQL source code Nikita Pettik
` (3 subsequent siblings)
9 siblings, 1 reply; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Also this patch resolves issue connected with wrong query plans during
select on spaces created from Lua: instead of index search in most cases
table scan was used. It appeared due to the fact that index was checked
on affinity compatibility with space format. So, if space is created
without affinity in format, indexes won't be used.
However, now all checks are related to field types, and as a result
query optimizer is able to choose correct index.
Closes #3886
Part of #3698
---
src/box/sql/build.c | 6 +-
src/box/sql/expr.c | 270 +++++++++++++++++----------------------
src/box/sql/fkey.c | 5 +-
src/box/sql/insert.c | 2 +-
src/box/sql/parse.y | 12 +-
src/box/sql/resolve.c | 4 +-
src/box/sql/select.c | 23 ++--
src/box/sql/sqliteInt.h | 46 ++++---
src/box/sql/vdbemem.c | 18 +--
src/box/sql/where.c | 25 ++--
src/box/sql/whereInt.h | 3 +-
src/box/sql/wherecode.c | 38 +++---
src/box/sql/whereexpr.c | 31 ++---
test/sql-tap/lua-tables.test.lua | 38 +++++-
test/sql/iproto.result | 2 +-
15 files changed, 257 insertions(+), 266 deletions(-)
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index b3f98c317..2885bb6d5 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -495,8 +495,12 @@ sql_affinity_to_field_type(enum affinity_type affinity)
return FIELD_TYPE_NUMBER;
case AFFINITY_TEXT:
return FIELD_TYPE_STRING;
- default:
+ case AFFINITY_BLOB:
return FIELD_TYPE_SCALAR;
+ case AFFINITY_UNDEFINED:
+ return FIELD_TYPE_ANY;
+ default:
+ unreachable();
}
}
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index c823c5a06..32606dac3 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -45,36 +45,12 @@ static void exprCodeBetween(Parse *, Expr *, int,
void (*)(Parse *, Expr *, int, int), int);
static int exprCodeVector(Parse * pParse, Expr * p, int *piToFree);
-char
-sqlite3TableColumnAffinity(struct space_def *def, int idx)
-{
- assert(idx < (int)def->field_count);
- return idx >= 0 ? def->fields[idx].affinity :
- AFFINITY_INTEGER;
-}
-
-/*
- * Return the 'affinity' of the expression pExpr if any.
- *
- * If pExpr is a column, a reference to a column via an 'AS' alias,
- * or a sub-select with a column as the return value, then the
- * affinity of that column is returned. Otherwise, 0x00 is returned,
- * indicating no affinity for the expression.
- *
- * i.e. the WHERE clause expressions in the following statements all
- * have an affinity:
- *
- * CREATE TABLE t1(a);
- * SELECT * FROM t1 WHERE a;
- * SELECT a AS b FROM t1 WHERE b;
- * SELECT * FROM t1 WHERE (select a from t1);
- */
-char
-sqlite3ExprAffinity(Expr * pExpr)
+enum field_type
+sql_expr_type(struct Expr *pExpr)
{
pExpr = sqlite3ExprSkipCollate(pExpr);
if (pExpr->flags & EP_Generic)
- return 0;
+ return FIELD_TYPE_ANY;
uint8_t op = pExpr->op;
struct ExprList *el;
if (op == TK_REGISTER)
@@ -83,27 +59,26 @@ sqlite3ExprAffinity(Expr * pExpr)
case TK_SELECT:
assert(pExpr->flags & EP_xIsSelect);
el = pExpr->x.pSelect->pEList;
- return sqlite3ExprAffinity(el->a[0].pExpr);
+ return sql_expr_type(el->a[0].pExpr);
case TK_CAST:
assert(!ExprHasProperty(pExpr, EP_IntValue));
- return pExpr->affinity;
+ return pExpr->type;
case TK_AGG_COLUMN:
case TK_COLUMN:
assert(pExpr->iColumn >= 0);
- return sqlite3TableColumnAffinity(pExpr->space_def,
- pExpr->iColumn);
+ return pExpr->space_def->fields[pExpr->iColumn].type;
case TK_SELECT_COLUMN:
assert(pExpr->pLeft->flags & EP_xIsSelect);
el = pExpr->pLeft->x.pSelect->pEList;
- return sqlite3ExprAffinity(el->a[pExpr->iColumn].pExpr);
+ return sql_expr_type(el->a[pExpr->iColumn].pExpr);
case TK_PLUS:
case TK_MINUS:
case TK_STAR:
case TK_SLASH:
assert(pExpr->pRight != NULL && pExpr->pLeft != NULL);
- enum affinity_type lhs_aff = sqlite3ExprAffinity(pExpr->pLeft);
- enum affinity_type rhs_aff = sqlite3ExprAffinity(pExpr->pRight);
- return sql_affinity_result(rhs_aff, lhs_aff);
+ enum field_type lhs_type = sql_expr_type(pExpr->pLeft);
+ enum field_type rhs_type = sql_expr_type(pExpr->pRight);
+ return sql_type_result(rhs_type, lhs_type);
case TK_LT:
case TK_GT:
case TK_EQ:
@@ -113,22 +88,22 @@ sqlite3ExprAffinity(Expr * pExpr)
case TK_AND:
case TK_OR:
/*
- * FIXME: should be changed to BOOL type or
- * affinity when it is implemented. Now simply
+ * FIXME: should be changed to BOOL type
+ * when it is implemented. Now simply
* return INTEGER.
*/
- return AFFINITY_INTEGER;
+ return FIELD_TYPE_INTEGER;
}
/*
* In case of unary plus we shouldn't discard
- * affinity of operand (since plus always features
- * NUMERIC affinity).
+ * type of operand (since plus always features
+ * NUMERIC type).
*/
if (op == TK_UPLUS) {
assert(pExpr->pRight == NULL);
- return pExpr->pLeft->affinity;
+ return pExpr->pLeft->type;
}
- return pExpr->affinity;
+ return pExpr->type;
}
/*
@@ -256,28 +231,29 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
return coll;
}
-enum affinity_type
-sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
+enum field_type
+sql_type_result(enum field_type lhs, enum field_type rhs)
{
- if (aff1 && aff2) {
- /* Both sides of the comparison are columns. If one has numeric
- * affinity, use that. Otherwise use no affinity.
+ if (lhs != FIELD_TYPE_ANY && rhs != FIELD_TYPE_ANY) {
+ /*
+ * Both sides of the comparison are columns.
+ * If one has numeric type, use that.
*/
- if (sqlite3IsNumericAffinity(aff1)
- || sqlite3IsNumericAffinity(aff2)) {
- return AFFINITY_REAL;
- } else {
- return AFFINITY_BLOB;
- }
- } else if (!aff1 && !aff2) {
- /* Neither side of the comparison is a column. Compare the
- * results directly.
+ if (sql_type_is_numeric(lhs) || sql_type_is_numeric(rhs))
+ return FIELD_TYPE_NUMBER;
+ else
+ return FIELD_TYPE_SCALAR;
+
+ } else if (lhs == FIELD_TYPE_ANY && rhs == FIELD_TYPE_ANY) {
+ /*
+ * Neither side of the comparison is a column.
+ * Compare the results directly.
*/
- return AFFINITY_BLOB;
+ return FIELD_TYPE_SCALAR;
} else {
- /* One side is a column, the other is not. Use the columns affinity. */
- assert(aff1 == 0 || aff2 == 0);
- return (aff1 + aff2);
+ if (lhs == FIELD_TYPE_ANY)
+ return rhs;
+ return lhs;
}
}
@@ -285,45 +261,44 @@ sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
* pExpr is a comparison operator. Return the type affinity that should
* be applied to both operands prior to doing the comparison.
*/
-static char
+static enum field_type
comparisonAffinity(Expr * pExpr)
{
- char aff;
assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
pExpr->op == TK_GT || pExpr->op == TK_GE || pExpr->op == TK_LE ||
pExpr->op == TK_NE);
assert(pExpr->pLeft);
- aff = sqlite3ExprAffinity(pExpr->pLeft);
+ enum field_type type = sql_expr_type(pExpr->pLeft);
if (pExpr->pRight) {
- enum affinity_type rhs_aff = sqlite3ExprAffinity(pExpr->pRight);
- aff = sql_affinity_result(rhs_aff, aff);
+ enum field_type rhs_type = sql_expr_type(pExpr->pRight);
+ type = sql_type_result(rhs_type, type);
} else if (ExprHasProperty(pExpr, EP_xIsSelect)) {
- enum affinity_type rhs_aff =
- sqlite3ExprAffinity(pExpr->x.pSelect->pEList->a[0].pExpr);
- aff = sql_affinity_result(rhs_aff, aff);
+ enum field_type rhs_type =
+ sql_expr_type(pExpr->x.pSelect->pEList->a[0].pExpr);
+ type = sql_type_result(rhs_type, type);
} else {
- aff = AFFINITY_BLOB;
+ type = FIELD_TYPE_SCALAR;
}
- return aff;
+ return type;
}
-/*
- * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
- * idx_affinity is the affinity of an indexed column. Return true
- * if the index with affinity idx_affinity may be used to implement
- * the comparison in pExpr.
+/**
+ * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
+ * @param idx_affinity is the affinity of an indexed column.
+ * @retval Return true if the index with @idx_type may be used to
+ * implement the comparison in expr.
*/
-int
-sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
+enum field_type
+sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type)
{
- char aff = comparisonAffinity(pExpr);
- switch (aff) {
- case AFFINITY_BLOB:
+ enum field_type type = comparisonAffinity(expr);
+ switch (type) {
+ case FIELD_TYPE_SCALAR:
return 1;
- case AFFINITY_TEXT:
- return idx_affinity == AFFINITY_TEXT;
+ case FIELD_TYPE_STRING:
+ return idx_type == FIELD_TYPE_STRING;
default:
- return sqlite3IsNumericAffinity(idx_affinity);
+ return sql_type_is_numeric(idx_type);
}
}
@@ -334,11 +309,11 @@ sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
static u8
binaryCompareP5(Expr * pExpr1, Expr * pExpr2, int jumpIfNull)
{
- enum affinity_type aff2 = sqlite3ExprAffinity(pExpr2);
- enum affinity_type aff1 = sqlite3ExprAffinity(pExpr1);
- enum affinity_type aff = sql_affinity_result(aff1, aff2) |
- (u8) jumpIfNull;
- return aff;
+ enum field_type lhs = sql_expr_type(pExpr2);
+ enum field_type rhs = sql_expr_type(pExpr1);
+ u8 type_mask = sql_field_type_to_affinity(sql_type_result(rhs, lhs)) |
+ (u8) jumpIfNull;
+ return type_mask;
}
int
@@ -2155,10 +2130,10 @@ sqlite3ExprCanBeNull(const Expr * p)
* answer.
*/
int
-sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
+sql_expr_needs_type_change(const Expr *p, enum field_type type)
{
u8 op;
- if (aff == AFFINITY_BLOB)
+ if (type == FIELD_TYPE_SCALAR)
return 1;
while (p->op == TK_UPLUS || p->op == TK_UMINUS) {
p = p->pLeft;
@@ -2167,27 +2142,21 @@ sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
if (op == TK_REGISTER)
op = p->op2;
switch (op) {
- case TK_INTEGER:{
- return aff == AFFINITY_INTEGER;
- }
- case TK_FLOAT:{
- return aff == AFFINITY_REAL;
- }
- case TK_STRING:{
- return aff == AFFINITY_TEXT;
- }
- case TK_BLOB:{
- return 1;
- }
+ case TK_INTEGER:
+ return type == FIELD_TYPE_INTEGER;
+ case TK_FLOAT:
+ return type == FIELD_TYPE_NUMBER;
+ case TK_STRING:
+ return type == FIELD_TYPE_STRING;
+ case TK_BLOB:
+ return 1;
case TK_COLUMN:{
- assert(p->iTable >= 0); /* p cannot be part of a CHECK constraint */
- return p->iColumn < 0
- && (aff == AFFINITY_INTEGER
- || aff == AFFINITY_REAL);
- }
- default:{
- return 0;
+ /* p cannot be part of a CHECK constraint. */
+ assert(p->iTable >= 0);
+ return p->iColumn < 0 && sql_type_is_numeric(type);
}
+ default:
+ return 0;
}
}
@@ -2436,14 +2405,14 @@ sqlite3FindInIndex(Parse * pParse, /* Parsing context */
Expr *pLhs = sqlite3VectorFieldSubexpr(pX->pLeft, i);
int iCol = pEList->a[i].pExpr->iColumn;
/* RHS table */
- enum affinity_type idxaff =
- sqlite3TableColumnAffinity(pTab->def, iCol);
- enum affinity_type lhs_aff = sqlite3ExprAffinity(pLhs);
+ assert(iCol >= 0);
+ enum field_type idx_type = pTab->def->fields[iCol].type;
+ enum field_type lhs_type = sql_expr_type(pLhs);
/*
* Index search is possible only if types
* of columns match.
*/
- if (lhs_aff != idxaff)
+ if (idx_type != lhs_type)
affinity_ok = 0;
}
@@ -2633,13 +2602,13 @@ exprINAffinity(Parse * pParse, Expr * pExpr)
int i;
for (i = 0; i < nVal; i++) {
Expr *pA = sqlite3VectorFieldSubexpr(pLeft, i);
- char a = sqlite3ExprAffinity(pA);
+ enum field_type lhs = sql_expr_type(pA);
if (pSelect) {
struct Expr *e = pSelect->pEList->a[i].pExpr;
- enum affinity_type aff = sqlite3ExprAffinity(e);
- zRet[i] = sql_affinity_result(aff, a);
+ enum field_type rhs = sql_expr_type(e);
+ zRet[i] = sql_field_type_to_affinity(sql_type_result(rhs, lhs));
} else {
- zRet[i] = a;
+ zRet[i] = sql_field_type_to_affinity(lhs);
}
}
zRet[nVal] = '\0';
@@ -2826,16 +2795,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
* that columns affinity when building index keys. If <expr> is not
* a column, use numeric affinity.
*/
- char affinity; /* Affinity of the LHS of the IN */
int i;
ExprList *pList = pExpr->x.pList;
struct ExprList_item *pItem;
int r1, r2, r3;
- affinity = sqlite3ExprAffinity(pLeft);
- if (!affinity) {
- affinity = AFFINITY_BLOB;
- }
+ enum field_type lhs_type =
+ sql_expr_type(pLeft);
+ if (lhs_type == FIELD_TYPE_ANY)
+ lhs_type = FIELD_TYPE_SCALAR;
bool unused;
sql_expr_coll(pParse, pExpr->pLeft,
&unused, &key_info->parts[0].coll_id);
@@ -2858,10 +2826,9 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
jmpIfDynamic = -1;
}
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
- char type =
- sql_affinity_to_field_type(affinity);
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
- 1, r2, &type,
+ 1, r2,
+ (char *) &lhs_type,
1);
sqlite3ExprCacheAffinityChange(pParse,
r3, 1);
@@ -3694,7 +3661,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
case TK_AGG_COLUMN:{
AggInfo *pAggInfo = pExpr->pAggInfo;
struct AggInfo_col *pCol = &pAggInfo->aCol[pExpr->iAgg];
- pExpr->affinity = pCol->pExpr->affinity;
+ pExpr->type = pCol->pExpr->type;
if (!pAggInfo->directMode) {
assert(pCol->iMem > 0);
return pCol->iMem;
@@ -3725,28 +3692,27 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
iTab = pParse->iSelfTab;
}
}
- pExpr->affinity =
- pExpr->space_def->fields[col].affinity;
+ pExpr->type = pExpr->space_def->fields[col].type;
return sqlite3ExprCodeGetColumn(pParse,
pExpr->space_def, col,
iTab, target,
pExpr->op2);
}
case TK_INTEGER:{
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
expr_code_int(pParse, pExpr, false, target);
return target;
}
#ifndef SQLITE_OMIT_FLOATING_POINT
case TK_FLOAT:{
- pExpr->affinity = AFFINITY_REAL;
+ pExpr->type = FIELD_TYPE_INTEGER;
assert(!ExprHasProperty(pExpr, EP_IntValue));
codeReal(v, pExpr->u.zToken, 0, target);
return target;
}
#endif
case TK_STRING:{
- pExpr->affinity = AFFINITY_TEXT;
+ pExpr->type = FIELD_TYPE_STRING;
assert(!ExprHasProperty(pExpr, EP_IntValue));
sqlite3VdbeLoadString(v, target, pExpr->u.zToken);
return target;
@@ -3764,7 +3730,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
assert(pExpr->u.zToken[0] == 'x'
|| pExpr->u.zToken[0] == 'X');
assert(pExpr->u.zToken[1] == '\'');
- pExpr->affinity = AFFINITY_BLOB;
+ pExpr->type = FIELD_TYPE_SCALAR;
z = &pExpr->u.zToken[2];
n = sqlite3Strlen30(z) - 1;
assert(z[n] == '\'');
@@ -3803,8 +3769,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
sqlite3VdbeAddOp2(v, OP_SCopy, inReg, target);
inReg = target;
}
- sqlite3VdbeAddOp2(v, OP_Cast, target,
- sql_affinity_to_field_type(pExpr->affinity));
+ sqlite3VdbeAddOp2(v, OP_Cast, target, pExpr->type);
testcase(usedAsColumnCache(pParse, inReg, inReg));
sqlite3ExprCacheAffinityChange(pParse, inReg, 1);
return inReg;
@@ -3847,7 +3812,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
testcase(regFree1 == 0);
testcase(regFree2 == 0);
}
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
break;
}
case TK_AND:
@@ -3892,14 +3857,14 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
testcase(regFree1 == 0);
testcase(regFree2 == 0);
if (op != TK_CONCAT)
- pExpr->affinity = AFFINITY_REAL;
+ pExpr->type = FIELD_TYPE_NUMBER;
else
- pExpr->affinity = AFFINITY_TEXT;
+ pExpr->type = FIELD_TYPE_STRING;
break;
}
case TK_UMINUS:{
Expr *pLeft = pExpr->pLeft;
- pExpr->affinity = AFFINITY_REAL;
+ pExpr->type = FIELD_TYPE_NUMBER;
assert(pLeft);
if (pLeft->op == TK_INTEGER) {
expr_code_int(pParse, pLeft, true, target);
@@ -3926,7 +3891,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
}
case TK_BITNOT:
case TK_NOT:{
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
assert(TK_BITNOT == OP_BitNot);
testcase(op == TK_BITNOT);
assert(TK_NOT == OP_Not);
@@ -3940,7 +3905,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
case TK_ISNULL:
case TK_NOTNULL:{
int addr;
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
assert(TK_ISNULL == OP_IsNull);
testcase(op == TK_ISNULL);
assert(TK_NOTNULL == OP_NotNull);
@@ -3964,9 +3929,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
"misuse of aggregate: %s()",
pExpr->u.zToken);
} else {
- enum field_type t =
- pInfo->aFunc->pFunc->ret_type;
- pExpr->affinity = sql_field_type_to_affinity(t);
+ pExpr->type = pInfo->aFunc->pFunc->ret_type;
return pInfo->aFunc[pExpr->iAgg].iMem;
}
break;
@@ -4003,18 +3966,15 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
break;
}
- if (pDef->ret_type != AFFINITY_UNDEFINED) {
- pExpr->affinity =
- sql_field_type_to_affinity(pDef->ret_type);
+ if (pDef->ret_type != FIELD_TYPE_ANY) {
+ pExpr->type = pDef->ret_type;
} else {
/*
* Otherwise, use first arg as
* expression affinity.
*/
- if (pFarg && pFarg->nExpr > 0) {
- pExpr->affinity =
- pFarg->a[0].pExpr->affinity;
- }
+ if (pFarg && pFarg->nExpr > 0)
+ pExpr->type = pFarg->a[0].pExpr->type;
}
/* Attempt a direct implementation of the built-in COALESCE() and
* IFNULL() functions. This avoids unnecessary evaluation of
@@ -4159,7 +4119,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
case TK_IN:{
int destIfFalse = sqlite3VdbeMakeLabel(v);
int destIfNull = sqlite3VdbeMakeLabel(v);
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
sqlite3VdbeAddOp2(v, OP_Null, 0, target);
sqlite3ExprCodeIN(pParse, pExpr, destIfFalse,
destIfNull);
@@ -4183,18 +4143,18 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
* Z is stored in pExpr->pList->a[1].pExpr.
*/
case TK_BETWEEN:{
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
exprCodeBetween(pParse, pExpr, target, 0, 0);
return target;
}
case TK_SPAN:
case TK_COLLATE:{
- pExpr->affinity = AFFINITY_TEXT;
+ pExpr->type = FIELD_TYPE_STRING;
return sqlite3ExprCodeTarget(pParse, pExpr->pLeft,
target);
}
case TK_UPLUS:{
- pExpr->affinity = AFFINITY_REAL;
+ pExpr->type = FIELD_TYPE_NUMBER;
return sqlite3ExprCodeTarget(pParse, pExpr->pLeft,
target);
}
@@ -4246,7 +4206,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
* floating point when extracting it from the record.
*/
if (pExpr->iColumn >= 0 && def->fields[
- pExpr->iColumn].affinity == AFFINITY_REAL) {
+ pExpr->iColumn].type == FIELD_TYPE_NUMBER) {
sqlite3VdbeAddOp1(v, OP_Realify, target);
}
#endif
diff --git a/src/box/sql/fkey.c b/src/box/sql/fkey.c
index a6a8a24dd..a3bcca26a 100644
--- a/src/box/sql/fkey.c
+++ b/src/box/sql/fkey.c
@@ -310,11 +310,10 @@ exprTableRegister(Parse * pParse, /* Parsing and code generating context */
if (pExpr) {
if (iCol >= 0) {
pExpr->iTable = regBase + iCol + 1;
- char affinity = pTab->def->fields[iCol].affinity;
- pExpr->affinity = affinity;
+ pExpr->type = pTab->def->fields[iCol].type;
} else {
pExpr->iTable = regBase;
- pExpr->affinity = AFFINITY_INTEGER;
+ pExpr->type = FIELD_TYPE_INTEGER;
}
}
return pExpr;
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index fd36e2786..acc7712f6 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -989,7 +989,7 @@ vdbe_emit_constraint_checks(struct Parse *parse_context, struct Table *tab,
if (part_count == 1) {
uint32_t fieldno = pk->def->key_def->parts[0].fieldno;
int reg_pk = new_tuple_reg + fieldno;
- if (def->fields[fieldno].affinity == AFFINITY_INTEGER) {
+ if (def->fields[fieldno].type == FIELD_TYPE_INTEGER) {
int skip_if_null = sqlite3VdbeMakeLabel(v);
if (autoinc_fieldno != UINT32_MAX) {
sqlite3VdbeAddOp2(v, OP_IsNull, reg_pk,
diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y
index 50bc25152..e71e0cdb5 100644
--- a/src/box/sql/parse.y
+++ b/src/box/sql/parse.y
@@ -818,16 +818,16 @@ idlist(A) ::= nm(Y).
memset(p, 0, sizeof(Expr));
switch (op) {
case TK_STRING:
- p->affinity = AFFINITY_TEXT;
+ p->type = FIELD_TYPE_STRING;
break;
case TK_BLOB:
- p->affinity = AFFINITY_BLOB;
+ p->type = FIELD_TYPE_SCALAR;
break;
case TK_INTEGER:
- p->affinity = AFFINITY_INTEGER;
+ p->type = FIELD_TYPE_INTEGER;
break;
case TK_FLOAT:
- p->affinity = AFFINITY_REAL;
+ p->type = FIELD_TYPE_NUMBER;
break;
}
p->op = (u8)op;
@@ -865,7 +865,7 @@ term(A) ::= FLOAT|BLOB(X). {spanExpr(&A,pParse,@X,X);/*A-overwrites-X*/}
term(A) ::= STRING(X). {spanExpr(&A,pParse,@X,X);/*A-overwrites-X*/}
term(A) ::= INTEGER(X). {
A.pExpr = sqlite3ExprAlloc(pParse->db, TK_INTEGER, &X, 1);
- A.pExpr->affinity = AFFINITY_INTEGER;
+ A.pExpr->type = FIELD_TYPE_INTEGER;
A.zStart = X.z;
A.zEnd = X.z + X.n;
if( A.pExpr ) A.pExpr->flags |= EP_Leaf;
@@ -898,7 +898,7 @@ expr(A) ::= expr(A) COLLATE id(C). {
expr(A) ::= CAST(X) LP expr(E) AS typedef(T) RP(Y). {
spanSet(&A,&X,&Y); /*A-overwrites-X*/
A.pExpr = sqlite3ExprAlloc(pParse->db, TK_CAST, 0, 1);
- A.pExpr->affinity = sql_field_type_to_affinity(T.type);
+ A.pExpr->type = T.type;
sqlite3ExprAttachSubtrees(pParse->db, A.pExpr, E.pExpr, 0);
}
%endif SQLITE_OMIT_CAST
diff --git a/src/box/sql/resolve.c b/src/box/sql/resolve.c
index c1253ab95..7f106457c 100644
--- a/src/box/sql/resolve.c
+++ b/src/box/sql/resolve.c
@@ -339,8 +339,8 @@ lookupName(Parse * pParse, /* The parsing context */
if (iCol < (int)pTab->def->field_count) {
cnt++;
if (iCol < 0) {
- pExpr->affinity =
- AFFINITY_INTEGER;
+ pExpr->type =
+ FIELD_TYPE_INTEGER;
} else if (pExpr->iTable == 0) {
testcase(iCol == 31);
testcase(iCol == 32);
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 0ee40093f..cc3e2f2fd 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1430,8 +1430,7 @@ sql_expr_list_to_key_info(struct Parse *parse, struct ExprList *list, int start)
sql_expr_coll(parse, item->pExpr, &unused, &id);
part->coll_id = id;
part->sort_order = item->sort_order;
- enum affinity_type aff = sqlite3ExprAffinity(item->pExpr);
- part->type =sql_affinity_to_field_type(aff);
+ part->type = sql_expr_type(item->pExpr);
}
return key_info;
}
@@ -1725,20 +1724,20 @@ generateColumnNames(Parse * pParse, /* Parser context */
p = pEList->a[i].pExpr;
if (NEVER(p == 0))
continue;
- switch (p->affinity) {
- case AFFINITY_INTEGER:
+ switch (p->type) {
+ case FIELD_TYPE_INTEGER:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "INTEGER",
SQLITE_TRANSIENT);
break;
- case AFFINITY_REAL:
+ case FIELD_TYPE_NUMBER:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "NUMERIC",
SQLITE_TRANSIENT);
break;
- case AFFINITY_TEXT:
+ case FIELD_TYPE_STRING:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "TEXT",
SQLITE_TRANSIENT);
break;
- case AFFINITY_BLOB:
+ case FIELD_TYPE_SCALAR:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BLOB",
SQLITE_TRANSIENT);
break;
@@ -1972,11 +1971,11 @@ sqlite3SelectAddColumnTypeAndCollation(Parse * pParse, /* Parsing contexts */
a = pSelect->pEList->a;
for (uint32_t i = 0; i < pTab->def->field_count; i++) {
p = a[i].pExpr;
- char affinity = sqlite3ExprAffinity(p);
- if (affinity == 0)
- affinity = AFFINITY_BLOB;
- pTab->def->fields[i].affinity = affinity;
- pTab->def->fields[i].type = sql_affinity_to_field_type(affinity);
+ enum field_type type = sql_expr_type(p);
+ if (type == FIELD_TYPE_ANY)
+ type = FIELD_TYPE_SCALAR;
+ pTab->def->fields[i].affinity = sql_field_type_to_affinity(type);
+ pTab->def->fields[i].type = type;
bool is_found;
uint32_t coll_id;
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 690fa6431..7d1159345 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1788,7 +1788,9 @@ struct Savepoint {
#define SAVEPOINT_RELEASE 1
#define SAVEPOINT_ROLLBACK 2
-#define sqlite3IsNumericAffinity(X) ((X)>=AFFINITY_INTEGER)
+#define sql_type_is_numeric(X) ((X) == FIELD_TYPE_INTEGER || \
+ (X) == FIELD_TYPE_NUMBER || \
+ (X) == FIELD_TYPE_UNSIGNED)
/*
* The AFFINITY_MASK values masks off the significant bits of an
@@ -2092,7 +2094,7 @@ struct Expr {
u8 op; /* Operation performed by this node */
union {
/** The affinity of the column. */
- enum affinity_type affinity;
+ enum field_type type;
/** Conflict action for RAISE() function. */
enum on_conflict_action on_conflict_action;
};
@@ -3707,7 +3709,9 @@ int sqlite3ExprIsConstantOrFunction(Expr *, u8);
int sqlite3ExprIsTableConstant(Expr *, int);
int sqlite3ExprIsInteger(Expr *, int *);
int sqlite3ExprCanBeNull(const Expr *);
-int sqlite3ExprNeedsNoAffinityChange(const Expr *, char);
+
+int
+sql_expr_needs_type_change(const Expr *epr, enum field_type type);
/**
* This routine generates VDBE code that causes a single row of a
@@ -4253,26 +4257,30 @@ sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
void
sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
-/**
- * Return superposition of two affinities.
- * This may be required for determining resulting
- * affinity of expressions like a + '2'.
- */
-enum affinity_type
-sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2);
+enum field_type
+sql_type_result(enum field_type lhs, enum field_type rhs);
-int sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity);
+enum field_type
+sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type);
/**
- * Return the affinity character for a single column of a table.
- * @param def space definition.
- * @param idx column index.
- * @retval AFFINITY
+ * Return the type of the expression pExpr.
+ *
+ * If pExpr is a column, a reference to a column via an 'AS' alias,
+ * or a sub-select with a column as the return value, then the
+ * type of that column is returned. Otherwise, type ANY is returned,
+ * indicating that the expression can feature any type.
+ *
+ * The WHERE clause expressions in the following statements all
+ * have an type:
+ *
+ * CREATE TABLE t1(a);
+ * SELECT * FROM t1 WHERE a;
+ * SELECT a AS b FROM t1 WHERE b;
+ * SELECT * FROM t1 WHERE (select a from t1);
*/
-char
-sqlite3TableColumnAffinity(struct space_def *def, int idx);
-
-char sqlite3ExprAffinity(Expr * pExpr);
+enum field_type
+sql_expr_type(Expr *pExpr);
/**
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index cd71641b0..d12a2a833 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -1279,11 +1279,10 @@ valueFromExpr(sqlite3 * db, /* The database connection */
assert((pExpr->flags & EP_TokenOnly) == 0 || pCtx == 0);
if (op == TK_CAST) {
- u8 aff = pExpr->affinity;
- rc = valueFromExpr(db, pExpr->pLeft, aff, ppVal, pCtx);
+ rc = valueFromExpr(db, pExpr->pLeft, pExpr->type, ppVal, pCtx);
testcase(rc != SQLITE_OK);
if (*ppVal) {
- sqlite3VdbeMemCast(*ppVal, aff);
+ sqlite3VdbeMemCast(*ppVal, pExpr->type);
sqlite3ValueApplyAffinity(*ppVal, affinity);
}
return rc;
@@ -1316,8 +1315,8 @@ valueFromExpr(sqlite3 * db, /* The database connection */
sqlite3ValueSetStr(pVal, -1, zVal, SQLITE_DYNAMIC);
}
if ((op == TK_INTEGER || op == TK_FLOAT)
- && affinity == AFFINITY_BLOB) {
- sqlite3ValueApplyAffinity(pVal, AFFINITY_REAL);
+ && affinity == FIELD_TYPE_SCALAR) {
+ sqlite3ValueApplyAffinity(pVal, FIELD_TYPE_INTEGER);
} else {
sqlite3ValueApplyAffinity(pVal, affinity);
}
@@ -1571,17 +1570,14 @@ sqlite3Stat4ProbeSetValue(Parse * pParse, /* Parse context */
alloc.pIdx = idx;
alloc.ppRec = ppRec;
- struct space *space = space_by_id(idx->space_id);
- assert(space != NULL);
for (i = 0; i < nElem; i++) {
sqlite3_value *pVal = 0;
Expr *pElem =
(pExpr ? sqlite3VectorFieldSubexpr(pExpr, i) : 0);
- u8 aff = sql_space_index_part_affinity(space->def, idx,
- iVal + i);
+ enum field_type type =
+ idx->key_def->parts[iVal + i].type;
alloc.iVal = iVal + i;
- aff = sql_affinity_to_field_type(aff);
- rc = stat4ValueFromExpr(pParse, pElem, aff, &alloc,
+ rc = stat4ValueFromExpr(pParse, pElem, type, &alloc,
&pVal);
if (!pVal)
break;
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 539296079..ac5390f92 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -298,7 +298,7 @@ whereScanNext(WhereScan * pScan)
/* Verify the affinity and collating sequence match */
if ((pTerm->eOperator & WO_ISNULL) == 0) {
pX = pTerm->pExpr;
- if (!sqlite3IndexAffinityOk(pX, pScan->idxaff))
+ if (!sql_index_type_is_ok(pX, pScan->idx_type))
continue;
if (pScan->is_column_seen) {
Parse *pParse =
@@ -367,7 +367,7 @@ whereScanInit(WhereScan * pScan, /* The WhereScan object being initialized */
pScan->pOrigWC = pWC;
pScan->pWC = pWC;
pScan->pIdxExpr = 0;
- pScan->idxaff = 0;
+ pScan->idx_type = FIELD_TYPE_ANY;
pScan->coll = NULL;
pScan->is_column_seen = false;
if (idx_def != NULL) {
@@ -381,9 +381,9 @@ whereScanInit(WhereScan * pScan, /* The WhereScan object being initialized */
struct space *sp = space_by_id(idx_def->space_id);
assert(sp != NULL);
if (sp->def->field_count == 0)
- pScan->idxaff = AFFINITY_BLOB;
+ pScan->idx_type = FIELD_TYPE_SCALAR;
else
- pScan->idxaff = sp->def->fields[iColumn].affinity;
+ pScan->idx_type = sp->def->fields[iColumn].type;
pScan->coll = idx_def->key_def->parts[j].coll;
pScan->is_column_seen = true;
} else {
@@ -424,13 +424,13 @@ where_scan_init(struct WhereScan *scan, struct WhereClause *clause,
scan->pOrigWC = clause;
scan->pWC = clause;
scan->pIdxExpr = NULL;
- scan->idxaff = 0;
+ scan->idx_type = FIELD_TYPE_ANY;
scan->coll = NULL;
scan->is_column_seen = false;
if (key_def != NULL) {
int j = column;
column = key_def->parts[j].fieldno;
- scan->idxaff = space_def->fields[column].affinity;
+ scan->idx_type = space_def->fields[column].type;
uint32_t coll_id = space_def->fields[column].coll_id;
struct coll_id *coll = coll_by_id(coll_id);
scan->coll = coll != NULL ? coll->coll : NULL;
@@ -2261,8 +2261,6 @@ whereRangeVectorLen(Parse * pParse, /* Parsing context */
/* Test if comparison i of pTerm is compatible with column (i+nEq)
* of the index. If not, exit the loop.
*/
- char aff; /* Comparison affinity */
- char idxaff = 0; /* Indexed columns affinity */
struct coll *pColl; /* Comparison collation sequence */
Expr *pLhs = pTerm->pExpr->pLeft->x.pList->a[i].pExpr;
Expr *pRhs = pTerm->pExpr->pRight;
@@ -2282,11 +2280,12 @@ whereRangeVectorLen(Parse * pParse, /* Parsing context */
pLhs->iColumn != (int)parts[i + nEq].fieldno ||
parts[i + nEq].sort_order != parts[nEq].sort_order)
break;
- enum affinity_type rhs_aff = sqlite3ExprAffinity(pRhs);
- aff = sql_affinity_result(rhs_aff, sqlite3ExprAffinity(pLhs));
- idxaff =
- sqlite3TableColumnAffinity(space->def, pLhs->iColumn);
- if (aff != idxaff)
+ enum field_type rhs_type = sql_expr_type(pRhs);
+ enum field_type type =
+ sql_type_result(rhs_type, sql_expr_type(pLhs));
+ enum field_type idx_type = pLhs->iColumn >= 0 ?
+ space->def->fields[pLhs->iColumn].type : FIELD_TYPE_INTEGER;
+ if (type != idx_type)
break;
uint32_t unused;
pColl = sql_binary_compare_coll_seq(pParse, pLhs, pRhs, &unused);
diff --git a/src/box/sql/whereInt.h b/src/box/sql/whereInt.h
index 4657055ea..286881a55 100644
--- a/src/box/sql/whereInt.h
+++ b/src/box/sql/whereInt.h
@@ -298,7 +298,8 @@ struct WhereScan {
/** Flag is set if actual column was encountered. */
bool is_column_seen;
Expr *pIdxExpr; /* Search for this index expression */
- char idxaff; /* Must match this affinity, if zCollName!=NULL */
+ /** Must match this type, if zCollName!=NULL */
+ enum field_type idx_type;
unsigned char nEquiv; /* Number of entries in aEquiv[] */
unsigned char iEquiv; /* Next unused slot in aEquiv[] */
u32 opMask; /* Acceptable operators */
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index aae5d6617..06335bcf7 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -392,6 +392,7 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
base++;
zAff++;
}
+
while (n > 1 && zAff[n - 1] == AFFINITY_BLOB) {
n--;
}
@@ -420,10 +421,11 @@ updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
{
int i;
for (i = 0; i < n; i++) {
+ enum field_type type = sql_affinity_to_field_type(zAff[i]);
Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
- enum affinity_type aff = sqlite3ExprAffinity(p);
- if (sql_affinity_result(aff, zAff[i]) == AFFINITY_BLOB
- || sqlite3ExprNeedsNoAffinityChange(p, zAff[i])) {
+ enum field_type expr_type = sql_expr_type(p);
+ if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
+ sql_expr_needs_type_change(p, type)) {
zAff[i] = AFFINITY_BLOB;
}
}
@@ -775,16 +777,16 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
VdbeCoverage(v);
}
if (zAff) {
- enum affinity_type aff =
- sqlite3ExprAffinity(pRight);
- if (sql_affinity_result(aff, zAff[j]) ==
- AFFINITY_BLOB) {
+ enum field_type type =
+ sql_expr_type(pRight);
+ enum field_type idx_type =
+ sql_affinity_to_field_type(zAff[j]);
+ if (sql_type_result(type, idx_type) ==
+ FIELD_TYPE_SCALAR) {
zAff[j] = AFFINITY_BLOB;
}
- if (sqlite3ExprNeedsNoAffinityChange
- (pRight, zAff[j])) {
+ if (sql_expr_needs_type_change(pRight, idx_type))
zAff[j] = AFFINITY_BLOB;
- }
}
}
}
@@ -1163,20 +1165,12 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
}
struct index_def *idx_pk = space->index[0]->def;
int fieldno = idx_pk->key_def->parts[0].fieldno;
- char affinity = is_format_set ?
- space->def->fields[fieldno].affinity :
- AFFINITY_BLOB;
- if (affinity == AFFINITY_UNDEFINED) {
- if (idx_pk->key_def->part_count == 1 &&
- space->def->fields[fieldno].type ==
- FIELD_TYPE_INTEGER)
- affinity = AFFINITY_INTEGER;
- else
- affinity = AFFINITY_BLOB;
- }
+ char fd_type = is_format_set ?
+ space->def->fields[fieldno].type :
+ FIELD_TYPE_ANY;
uint32_t pk_part_count = idx_pk->key_def->part_count;
- if (pk_part_count == 1 && affinity == AFFINITY_INTEGER) {
+ if (pk_part_count == 1 && fd_type == FIELD_TYPE_INTEGER) {
/* Right now INTEGER PRIMARY KEY is the only option to
* get Tarantool's INTEGER column type. Need special handling
* here: try to loosely convert FLOAT to INT. If RHS type
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 342064ec8..6ffd5ae84 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -269,7 +269,7 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
pLeft = pList->a[1].pExpr;
/* Value might be numeric */
if (pLeft->op != TK_COLUMN ||
- sqlite3ExprAffinity(pLeft) != AFFINITY_TEXT) {
+ sql_expr_type(pLeft) != FIELD_TYPE_STRING) {
/* IMP: R-02065-49465 The left-hand side of the
* LIKE operator must be the name of an indexed
* column with TEXT affinity.
@@ -748,19 +748,18 @@ exprAnalyzeOrTerm(SrcList * pSrc, /* the FROM clause */
} else if (pOrTerm->u.leftColumn != iColumn) {
okToChngToIN = 0;
} else {
- int affLeft, affRight;
/* If the right-hand side is also a column, then the affinities
* of both right and left sides must be such that no type
* conversions are required on the right. (Ticket #2249)
*/
- affRight =
- sqlite3ExprAffinity(pOrTerm->pExpr->
- pRight);
- affLeft =
- sqlite3ExprAffinity(pOrTerm->pExpr->
- pLeft);
- if (affRight != 0
- && affRight != affLeft) {
+ enum field_type rhs =
+ sql_expr_type(pOrTerm->pExpr->
+ pRight);
+ enum field_type lhs =
+ sql_expr_type(pOrTerm->pExpr->
+ pLeft);
+ if (rhs != FIELD_TYPE_ANY &&
+ rhs != lhs) {
okToChngToIN = 0;
} else {
pOrTerm->wtFlags |= TERM_OR_OK;
@@ -836,21 +835,17 @@ exprAnalyzeOrTerm(SrcList * pSrc, /* the FROM clause */
static int
termIsEquivalence(Parse * pParse, Expr * pExpr)
{
- char aff1, aff2;
if (!OptimizationEnabled(pParse->db, SQLITE_Transitive))
return 0;
if (pExpr->op != TK_EQ)
return 0;
if (ExprHasProperty(pExpr, EP_FromJoin))
return 0;
- aff1 = sqlite3ExprAffinity(pExpr->pLeft);
- aff2 = sqlite3ExprAffinity(pExpr->pRight);
- if (aff1 != aff2
- && (!sqlite3IsNumericAffinity(aff1)
- || !sqlite3IsNumericAffinity(aff2))
- ) {
+ enum field_type lhs_type = sql_expr_type(pExpr->pLeft);
+ enum field_type rhs_type = sql_expr_type(pExpr->pRight);
+ if (lhs_type != rhs_type && (!sql_type_is_numeric(lhs_type) ||
+ !sql_type_is_numeric(rhs_type)))
return 0;
- }
uint32_t unused;
struct coll *coll1 =
sql_binary_compare_coll_seq(pParse, pExpr->pLeft, pExpr->pRight,
diff --git a/test/sql-tap/lua-tables.test.lua b/test/sql-tap/lua-tables.test.lua
index 19d80b611..409976bdd 100755
--- a/test/sql-tap/lua-tables.test.lua
+++ b/test/sql-tap/lua-tables.test.lua
@@ -1,6 +1,6 @@
#!/usr/bin/env tarantool
test = require("sqltester")
-test:plan(9)
+test:plan(12)
test:do_test(
"lua-tables-prepare-1",
@@ -124,4 +124,40 @@ test:do_execsql_test(
[[SELECT * FROM "t" INDEXED BY "i"]],
{1, 4, 2, 2, 3, 3, 4, 3})
+-- gh-3886: indexes created from Lua are visible
+-- to query optimizer.
+--
+test:do_test(
+ "lua-tables-prepare-10",
+ function()
+ sp = box.schema.space.create("TEST", {
+ engine = 'memtx',
+ format = {
+ { name = 'ID', type = 'unsigned' },
+ { name = 'A', type = 'unsigned' }
+ }})
+ sp:create_index('primary', {parts = {1, 'unsigned' } })
+ sp:create_index('secondary', {parts = {2, 'unsigned' } })
+ sp:insert({1,1})
+ sp:insert({2,2})
+ sp:insert({3,3})
+ end,
+ {})
+
+test:do_eqp_test(
+ 11,
+ [[
+ SELECT * FROM test WHERE id = 2;
+ ]], {
+ {0, 0, 0, 'SEARCH TABLE TEST USING PRIMARY KEY (ID=?)'}
+ })
+
+test:do_eqp_test(
+ 12,
+ [[
+ SELECT * FROM test WHERE a = 5;
+ ]], {
+ {0, 0, 0, 'SEARCH TABLE TEST USING COVERING INDEX secondary (A=?)'}
+ })
+
test:finish_test()
diff --git a/test/sql/iproto.result b/test/sql/iproto.result
index 9ace282d5..326eda586 100644
--- a/test/sql/iproto.result
+++ b/test/sql/iproto.result
@@ -792,7 +792,7 @@ cn:execute('select * from "test"')
---
- metadata:
- name: id
- type: UNKNOWN
+ type: INTEGER
- name: x
type: UNKNOWN
rows:
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2018-12-28 9:34 ` [tarantool-patches] [PATCH 6/8] sql: replace affinity with field type in struct Expr Nikita Pettik
@ 2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-16 14:26 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2018-12-29 17:42 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 10 comments below.
On 28/12/2018 12:34, Nikita Pettik wrote:
> Also this patch resolves issue connected with wrong query plans during
> select on spaces created from Lua: instead of index search in most cases
> table scan was used. It appeared due to the fact that index was checked
> on affinity compatibility with space format. So, if space is created
> without affinity in format, indexes won't be used.
> However, now all checks are related to field types, and as a result
> query optimizer is able to choose correct index.
>
> Closes #3886
> Part of #3698
> ---
> src/box/sql/build.c | 6 +-
> src/box/sql/expr.c | 270 +++++++++++++++++----------------------
> src/box/sql/fkey.c | 5 +-
> src/box/sql/insert.c | 2 +-
> src/box/sql/parse.y | 12 +-
> src/box/sql/resolve.c | 4 +-
> src/box/sql/select.c | 23 ++--
> src/box/sql/sqliteInt.h | 46 ++++---
> src/box/sql/vdbemem.c | 18 +--
> src/box/sql/where.c | 25 ++--
> src/box/sql/whereInt.h | 3 +-
> src/box/sql/wherecode.c | 38 +++---
> src/box/sql/whereexpr.c | 31 ++---
> test/sql-tap/lua-tables.test.lua | 38 +++++-
> test/sql/iproto.result | 2 +-
> 15 files changed, 257 insertions(+), 266 deletions(-)
>
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index c823c5a06..32606dac3 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -256,28 +231,29 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
> return coll;
> }
>
> -enum affinity_type
> -sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
> +enum field_type
> +sql_type_result(enum field_type lhs, enum field_type rhs)
> {
> - if (aff1 && aff2) {
> - /* Both sides of the comparison are columns. If one has numeric
> - * affinity, use that. Otherwise use no affinity.
> + if (lhs != FIELD_TYPE_ANY && rhs != FIELD_TYPE_ANY) {
> + /*
> + * Both sides of the comparison are columns.
> + * If one has numeric type, use that.
> */
1. The comment is useless IMO. It just repeats exactly the
same obvious thing what is written in the next line.
Writing comments, keep in mind our meme:
https://github.com/Gerold103/tarantool-memes/blob/master/CAPTAIN.png
Also, I've just understood that we can post memes as a links to
my meme repository! We are saved!
> - if (sqlite3IsNumericAffinity(aff1)
> - || sqlite3IsNumericAffinity(aff2)) {
> - return AFFINITY_REAL;
> - } else {
> - return AFFINITY_BLOB;
> - }
> - } else if (!aff1 && !aff2) {
> - /* Neither side of the comparison is a column. Compare the
> - * results directly.
> + if (sql_type_is_numeric(lhs) || sql_type_is_numeric(rhs))
> + return FIELD_TYPE_NUMBER;
> + else
> + return FIELD_TYPE_SCALAR;
> +
2. Unnecessary empty line.
> + } else if (lhs == FIELD_TYPE_ANY && rhs == FIELD_TYPE_ANY) {
> + /*
> + * Neither side of the comparison is a column.
> + * Compare the results directly.
> */
> - return AFFINITY_BLOB;
> + return FIELD_TYPE_SCALAR;
> } else {
> - /* One side is a column, the other is not. Use the columns affinity. */
> - assert(aff1 == 0 || aff2 == 0);
> - return (aff1 + aff2);
> + if (lhs == FIELD_TYPE_ANY)
> + return rhs;
> + return lhs;
> }
> }
>
> @@ -285,45 +261,44 @@ sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
> * pExpr is a comparison operator. Return the type affinity that should
> * be applied to both operands prior to doing the comparison.
> */
> -static char
> +static enum field_type
> comparisonAffinity(Expr * pExpr)
3. But it is not affinity anymore.
> {
> - char aff;
> assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
> pExpr->op == TK_GT || pExpr->op == TK_GE || pExpr->op == TK_LE ||
> pExpr->op == TK_NE);
> assert(pExpr->pLeft);
> - aff = sqlite3ExprAffinity(pExpr->pLeft);
> + enum field_type type = sql_expr_type(pExpr->pLeft);
> if (pExpr->pRight) {
> - enum affinity_type rhs_aff = sqlite3ExprAffinity(pExpr->pRight);
> - aff = sql_affinity_result(rhs_aff, aff);
> + enum field_type rhs_type = sql_expr_type(pExpr->pRight);
> + type = sql_type_result(rhs_type, type);
> } else if (ExprHasProperty(pExpr, EP_xIsSelect)) {
> - enum affinity_type rhs_aff =
> - sqlite3ExprAffinity(pExpr->x.pSelect->pEList->a[0].pExpr);
> - aff = sql_affinity_result(rhs_aff, aff);
> + enum field_type rhs_type =
> + sql_expr_type(pExpr->x.pSelect->pEList->a[0].pExpr);
> + type = sql_type_result(rhs_type, type);
> } else {
> - aff = AFFINITY_BLOB;
> + type = FIELD_TYPE_SCALAR;
> }
> - return aff;
> + return type;
> }
>
> -/*
> - * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
> - * idx_affinity is the affinity of an indexed column. Return true
> - * if the index with affinity idx_affinity may be used to implement
> - * the comparison in pExpr.
> +/**
> + * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
> + * @param idx_affinity is the affinity of an indexed column.
> + * @retval Return true if the index with @idx_type may be used to
> + * implement the comparison in expr.
> */
> -int
> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
> +enum field_type
> +sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type)
4. Strictly speaking, it is not idx_type. It is a type of one
column. I think, it should be renamed to just type, or field_type,
as well as in struct WhereScan.
Also, sorry, but I hate this suffix '_ok' or 'Ok'. It just says
nothing about what this function does. As well as that it has
nothing to do with an index, but has 'index' in the name.
Maybe rename to something like 'sql_expr_cmp_type_is_possible' ?
> {
> - char aff = comparisonAffinity(pExpr);
> - switch (aff) {
> - case AFFINITY_BLOB:
> + enum field_type type = comparisonAffinity(expr);
> + switch (type) {
> + case FIELD_TYPE_SCALAR:
> return 1;
> - case AFFINITY_TEXT:
> - return idx_affinity == AFFINITY_TEXT;
> + case FIELD_TYPE_STRING:
> + return idx_type == FIELD_TYPE_STRING;
> default:
> - return sqlite3IsNumericAffinity(idx_affinity);
> + return sql_type_is_numeric(idx_type);
> }
> }
>
> @@ -2155,10 +2130,10 @@ sqlite3ExprCanBeNull(const Expr * p)
> * answer.
> */
> int
> -sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
> +sql_expr_needs_type_change(const Expr *p, enum field_type type)
5. Name is changed from negative to positive, but the logic is
the same, why? Original was 'NoChange', now it is 'needs_change'.
Also, please, move the function comment to the declaration in sqliteInt.h.
> {
> u8 op;
> - if (aff == AFFINITY_BLOB)
> + if (type == FIELD_TYPE_SCALAR)
> return 1;
> while (p->op == TK_UPLUS || p->op == TK_UMINUS) {
> p = p->pLeft;
> @@ -2167,27 +2142,21 @@ sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
> if (op == TK_REGISTER)
> op = p->op2;
> switch (op) {
> - case TK_INTEGER:{
> - return aff == AFFINITY_INTEGER;
> - }
> - case TK_FLOAT:{
> - return aff == AFFINITY_REAL;
> - }
> - case TK_STRING:{
> - return aff == AFFINITY_TEXT;
> - }
> - case TK_BLOB:{
> - return 1;
> - }
> + case TK_INTEGER:
> + return type == FIELD_TYPE_INTEGER;
> + case TK_FLOAT:
> + return type == FIELD_TYPE_NUMBER;
> + case TK_STRING:
> + return type == FIELD_TYPE_STRING;
> + case TK_BLOB:
> + return 1;
> case TK_COLUMN:{
> - assert(p->iTable >= 0); /* p cannot be part of a CHECK constraint */
> - return p->iColumn < 0
> - && (aff == AFFINITY_INTEGER
> - || aff == AFFINITY_REAL);
> - }
> - default:{
> - return 0;
> + /* p cannot be part of a CHECK constraint. */
> + assert(p->iTable >= 0);
> + return p->iColumn < 0 && sql_type_is_numeric(type);
> }
6. Please, remove these braces too, if you decided to refactor
sqlite3ExprNeedsNoAffinityChange.
> + default:
> + return 0;
> }
> }
>
> @@ -2826,16 +2795,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
> * that columns affinity when building index keys. If <expr> is not
> * a column, use numeric affinity.
> */
> - char affinity; /* Affinity of the LHS of the IN */
> int i;
> ExprList *pList = pExpr->x.pList;
> struct ExprList_item *pItem;
> int r1, r2, r3;
>
> - affinity = sqlite3ExprAffinity(pLeft);
> - if (!affinity) {
> - affinity = AFFINITY_BLOB;
> - }
> + enum field_type lhs_type =
> + sql_expr_type(pLeft);
> + if (lhs_type == FIELD_TYPE_ANY)
> + lhs_type = FIELD_TYPE_SCALAR;
7. Why did not you move this conversion to sql_expr_type? I mean
ANY -> SCALAR. ANY differs from SCALAR in only one thing - it is
able to store MP_MAP and MP_ARRAY. So I am slightly bent upon
frequency of ANY usage in SQL, wherein MAP/ARRAY does not exist.
> bool unused;
> sql_expr_coll(pParse, pExpr->pLeft,
> &unused, &key_info->parts[0].coll_id);
> diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
> index 690fa6431..7d1159345 100644
> --- a/src/box/sql/sqliteInt.h
> +++ b/src/box/sql/sqliteInt.h
> @@ -2092,7 +2094,7 @@ struct Expr {
> u8 op; /* Operation performed by this node */
> union {
> /** The affinity of the column. */
8. Not affinity anymore.
> - enum affinity_type affinity;
> + enum field_type type;
> /** Conflict action for RAISE() function. */
> enum on_conflict_action on_conflict_action;
> };
> @@ -4253,26 +4257,30 @@ sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
> void
> sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
>
> -/**
> - * Return superposition of two affinities.
> - * This may be required for determining resulting
> - * affinity of expressions like a + '2'.
> - */
> -enum affinity_type
> -sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2);
> +enum field_type
> +sql_type_result(enum field_type lhs, enum field_type rhs);
>
> -int sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity);
> +enum field_type
> +sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type);
>
> /**
> - * Return the affinity character for a single column of a table.
> - * @param def space definition.
> - * @param idx column index.
> - * @retval AFFINITY
> + * Return the type of the expression pExpr.
> + *
> + * If pExpr is a column, a reference to a column via an 'AS' alias,
> + * or a sub-select with a column as the return value, then the
> + * type of that column is returned. Otherwise, type ANY is returned,
> + * indicating that the expression can feature any type.
9. Returning to point 7 - I do not think it should return ANY. Anyway
you convert it to SCALAR in all places.
> + *
> + * The WHERE clause expressions in the following statements all
> + * have an type:
> + *
> + * CREATE TABLE t1(a);
> + * SELECT * FROM t1 WHERE a;
> + * SELECT a AS b FROM t1 WHERE b;
> + * SELECT * FROM t1 WHERE (select a from t1);
> */
> -char
> -sqlite3TableColumnAffinity(struct space_def *def, int idx);
> -
> -char sqlite3ExprAffinity(Expr * pExpr);
> +enum field_type
> +sql_expr_type(Expr *pExpr);
>
>
> /**
> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
> index aae5d6617..06335bcf7 100644
> --- a/src/box/sql/wherecode.c
> +++ b/src/box/sql/wherecode.c
> @@ -392,6 +392,7 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
> base++;
> zAff++;
> }
> +
> while (n > 1 && zAff[n - 1] == AFFINITY_BLOB) {
10. Garbage diff.
> n--;
> }
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-01-16 14:26 ` n.pettik
2019-01-22 15:41 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-16 14:26 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> -enum affinity_type
>> -sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
>> +enum field_type
>> +sql_type_result(enum field_type lhs, enum field_type rhs)
>> {
>> - if (aff1 && aff2) {
>> - /* Both sides of the comparison are columns. If one has numeric
>> - * affinity, use that. Otherwise use no affinity.
>> + if (lhs != FIELD_TYPE_ANY && rhs != FIELD_TYPE_ANY) {
>> + /*
>> + * Both sides of the comparison are columns.
>> + * If one has numeric type, use that.
>> */
>
> 1. The comment is useless IMO. It just repeats exactly the
> same obvious thing what is written in the next line.
Ok, I’d rather agree. I simply copied that comment from
original SQLite function. Now removed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 4816310c3..ba7269d08 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -235,15 +235,10 @@ enum field_type
sql_type_result(enum field_type lhs, enum field_type rhs)
{
if (lhs != FIELD_TYPE_ANY && rhs != FIELD_TYPE_ANY) {
- /*
- * Both sides of the comparison are columns.
- * If one has numeric type, use that.
- */
>
>> - if (sqlite3IsNumericAffinity(aff1)
>> - || sqlite3IsNumericAffinity(aff2)) {
>> - return AFFINITY_REAL;
>> - } else {
>> - return AFFINITY_BLOB;
>> - }
>> - } else if (!aff1 && !aff2) {
>> - /* Neither side of the comparison is a column. Compare the
>> - * results directly.
>> + if (sql_type_is_numeric(lhs) || sql_type_is_numeric(rhs))
>> + return FIELD_TYPE_NUMBER;
>> + else
>> + return FIELD_TYPE_SCALAR;
>> +
>
> 2. Unnecessary empty line.
Removed:
if (sql_type_is_numeric(lhs) || sql_type_is_numeric(rhs))
return FIELD_TYPE_NUMBER;
else
return FIELD_TYPE_SCALAR;
-
} else if (lhs == FIELD_TYPE_ANY && rhs == FIELD_TYPE_ANY) {
/*
* Neither side of the comparison is a column.
>
>> + } else if (lhs == FIELD_TYPE_ANY && rhs == FIELD_TYPE_ANY) {
>> + /*
>> + * Neither side of the comparison is a column.
>> + * Compare the results directly.
>> */
>> - return AFFINITY_BLOB;
>> + return FIELD_TYPE_SCALAR;
>> } else {
>> - /* One side is a column, the other is not. Use the columns affinity. */
>> - assert(aff1 == 0 || aff2 == 0);
>> - return (aff1 + aff2);
>> + if (lhs == FIELD_TYPE_ANY)
>> + return rhs;
>> + return lhs;
>> }
>> }
>> @@ -285,45 +261,44 @@ sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2)
>> * pExpr is a comparison operator. Return the type affinity that should
>> * be applied to both operands prior to doing the comparison.
>> */
>> -static char
>> +static enum field_type
>> comparisonAffinity(Expr * pExpr)
>
> 3. But it is not affinity anymore.
Renamed:
-/*
- * pExpr is a comparison operator. Return the type affinity that should
- * be applied to both operands prior to doing the comparison.
+/**
+ * pExpr is a comparison operator. Return the type affinity
+ * that should be applied to both operands prior to doing
+ * the comparison.
*/
static enum field_type
-comparisonAffinity(Expr * pExpr)
+comparison_mutual_type(struct Expr *pExpr)
{
assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
pExpr->op == TK_GT || pExpr->op == TK_GE || pExpr->op == TK_LE ||
>> -/*
>> - * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
>> - * idx_affinity is the affinity of an indexed column. Return true
>> - * if the index with affinity idx_affinity may be used to implement
>> - * the comparison in pExpr.
>> +/**
>> + * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
>> + * @param idx_affinity is the affinity of an indexed column.
>> + * @retval Return true if the index with @idx_type may be used to
>> + * implement the comparison in expr.
>> */
>> -int
>> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
>> +enum field_type
>> +sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type)
>
> 4. Strictly speaking, it is not idx_type. It is a type of one
> column. I think, it should be renamed to just type, or field_type,
> as well as in struct WhereScan.
Yea, it’s fair. Renamed:
-/*
- * pExpr is a comparison operator. Return the type affinity that should
- * be applied to both operands prior to doing the comparison.
+/**
+ * pExpr is a comparison operator. Return the type affinity
+ * that should be applied to both operands prior to doing
+ * the comparison.
*/
static enum field_type
-comparisonAffinity(Expr * pExpr)
+expr_cmp_mutual_type(struct Expr *pExpr)
{
assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
pExpr->op == TK_GT || pExpr->op == TK_GE || pExpr->op == TK_LE ||
> Also, sorry, but I hate this suffix '_ok' or 'Ok'. It just says
> nothing about what this function does. As well as that it has
> nothing to do with an index, but has 'index' in the name.
>
> Maybe rename to something like 'sql_expr_cmp_type_is_possible’ ?
Done:
@@ -289,9 +285,9 @@ comparisonAffinity(Expr * pExpr)
* implement the comparison in expr.
*/
enum field_type
-sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type)
+sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type)
{
- enum field_type type = comparisonAffinity(expr);
+ enum field_type type = expr_cmp_mutual_type(expr);
switch (type) {
case FIELD_TYPE_SCALAR:
return 1;
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 216e8d57f..7116f34c5 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -4261,7 +4261,7 @@ enum field_type
sql_type_result(enum field_type lhs, enum field_type rhs);
enum field_type
-sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type);
+sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type);
/**
* Return the type of the expression pExpr.
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 54531a8a5..a2e55e8ac 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -298,7 +298,7 @@ whereScanNext(WhereScan * pScan)
/* Verify the affinity and collating sequence match */
if ((pTerm->eOperator & WO_ISNULL) == 0) {
pX = pTerm->pExpr;
- if (!sql_index_type_is_ok(pX, pScan->idx_type))
+ if (!sql_expr_cmp_type_is_compatible(pX, pScan->idx_type))
continue;
if (pScan->is_column_seen) {
Parse *pParse =
>
>> {
>> - char aff = comparisonAffinity(pExpr);
>> - switch (aff) {
>> - case AFFINITY_BLOB:
>> + enum field_type type = comparisonAffinity(expr);
>> + switch (type) {
>> + case FIELD_TYPE_SCALAR:
>> return 1;
>> - case AFFINITY_TEXT:
>> - return idx_affinity == AFFINITY_TEXT;
>> + case FIELD_TYPE_STRING:
>> + return idx_type == FIELD_TYPE_STRING;
>> default:
>> - return sqlite3IsNumericAffinity(idx_affinity);
>> + return sql_type_is_numeric(idx_type);
>> }
>> }
>> @@ -2155,10 +2130,10 @@ sqlite3ExprCanBeNull(const Expr * p)
>> * answer.
>> */
>> int
>> -sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
>> +sql_expr_needs_type_change(const Expr *p, enum field_type type)
>
> 5. Name is changed from negative to positive, but the logic is
> the same, why? Original was 'NoChange', now it is 'needs_change’.
My fault, missed ’no’ particle. Fixed:
-/*
- * Return TRUE if the given expression is a constant which would be
- * unchanged by OP_ApplyType with the type given in the second
- * argument.
- *
- * This routine is used to determine if the OP_ApplyType operation
- * can be omitted. When in doubt return FALSE. A false negative
- * is harmless. A false positive, however, can result in the wrong
- * answer.
- */
int
-sql_expr_needs_type_change(const Expr *p, enum field_type type)
+sql_expr_needs_no_type_change(const Expr *p, enum field_type type)
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 216e8d57f..5541cb6b2 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3710,8 +3710,18 @@ int sqlite3ExprIsTableConstant(Expr *, int);
int sqlite3ExprIsInteger(Expr *, int *);
int sqlite3ExprCanBeNull(const Expr *);
+/**
+ * Return TRUE if the given expression is a constant which would
+ * be unchanged by OP_ApplyType with the type given in the second
+ * argument.
+ *
+ * This routine is used to determine if the OP_ApplyType operation
+ * can be omitted. When in doubt return FALSE. A false negative
+ * is harmless. A false positive, however, can result in the wrong
+ * answer.
+ */
int
-sql_expr_needs_type_change(const Expr *epr, enum field_type type);
+sql_expr_needs_no_type_change(const Expr *epr, enum field_type type);
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index fe5c56c07..98b55db7a 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -428,7 +428,7 @@ updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
enum field_type expr_type = sql_expr_type(p);
if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
- sql_expr_needs_type_change(p, type)) {
+ sql_expr_needs_no_type_change(p, type)) {
zAff[i] = AFFINITY_BLOB;
}
}
@@ -788,7 +788,7 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
FIELD_TYPE_SCALAR) {
zAff[j] = AFFINITY_BLOB;
}
- if (sql_expr_needs_type_change(pRight, idx_type))
+ if (sql_expr_needs_no_type_change(pRight, idx_type))
zAff[j] = AFFINITY_BLOB;
}
> Also, please, move the function comment to the declaration in sqliteInt.h.
Done, see above.
>
>> {
>> u8 op;
>> - if (aff == AFFINITY_BLOB)
>> + if (type == FIELD_TYPE_SCALAR)
>> return 1;
>> while (p->op == TK_UPLUS || p->op == TK_UMINUS) {
>> p = p->pLeft;
>> @@ -2167,27 +2142,21 @@ sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
>> if (op == TK_REGISTER)
>> op = p->op2;
>> switch (op) {
>> - case TK_INTEGER:{
>> - return aff == AFFINITY_INTEGER;
>> - }
>> - case TK_FLOAT:{
>> - return aff == AFFINITY_REAL;
>> - }
>> - case TK_STRING:{
>> - return aff == AFFINITY_TEXT;
>> - }
>> - case TK_BLOB:{
>> - return 1;
>> - }
>> + case TK_INTEGER:
>> + return type == FIELD_TYPE_INTEGER;
>> + case TK_FLOAT:
>> + return type == FIELD_TYPE_NUMBER;
>> + case TK_STRING:
>> + return type == FIELD_TYPE_STRING;
>> + case TK_BLOB:
>> + return 1;
>> case TK_COLUMN:{
>> - assert(p->iTable >= 0); /* p cannot be part of a CHECK constraint */
>> - return p->iColumn < 0
>> - && (aff == AFFINITY_INTEGER
>> - || aff == AFFINITY_REAL);
>> - }
>> - default:{
>> - return 0;
>> + /* p cannot be part of a CHECK constraint. */
>> + assert(p->iTable >= 0);
>> + return p->iColumn < 0 && sql_type_is_numeric(type);
>> }
>
> 6. Please, remove these braces too, if you decided to refactor
> sqlite3ExprNeedsNoAffinityChange.
Ok:
@@ -2150,11 +2136,10 @@ sql_expr_needs_type_change(const Expr *p, enum field_type type)
return type == FIELD_TYPE_STRING;
case TK_BLOB:
return 1;
- case TK_COLUMN:{
- /* p cannot be part of a CHECK constraint. */
- assert(p->iTable >= 0);
- return p->iColumn < 0 && sql_type_is_numeric(type);
- }
+ case TK_COLUMN:
+ /* p cannot be part of a CHECK constraint. */
+ assert(p->iTable >= 0);
+ return p->iColumn < 0 && sql_type_is_numeric(type);
default:
>
>> + default:
>> + return 0;
>> }
>> }
>> @@ -2826,16 +2795,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>> * that columns affinity when building index keys. If <expr> is not
>> * a column, use numeric affinity.
>> */
>> - char affinity; /* Affinity of the LHS of the IN */
>> int i;
>> ExprList *pList = pExpr->x.pList;
>> struct ExprList_item *pItem;
>> int r1, r2, r3;
>> - affinity = sqlite3ExprAffinity(pLeft);
>> - if (!affinity) {
>> - affinity = AFFINITY_BLOB;
>> - }
>> + enum field_type lhs_type =
>> + sql_expr_type(pLeft);
>> + if (lhs_type == FIELD_TYPE_ANY)
>> + lhs_type = FIELD_TYPE_SCALAR;
>
> 7. Why did not you move this conversion to sql_expr_type? I mean
> ANY -> SCALAR.
Unfortunately, right now I can’t replace it with ANY. It is still used
when IN operator comes with one operand (see EP_Generic flag).
I have already sent patch which removes it:
https://github.com/tarantool/tarantool/tree/np/gh-3934-IN-operator-fix
After it hits 2.1, I will replace SCALAR with ANY.
> ANY differs from SCALAR in only one thing - it is
> able to store MP_MAP and MP_ARRAY. So I am slightly bent upon
> frequency of ANY usage in SQL, wherein MAP/ARRAY does not exist.
But still we can SELECT data from spaces created from Lua.
For instance, _fk_constraint features type ARRAY in its format,
so we can’t ignore this type even now (IMHO).
>
>> bool unused;
>> sql_expr_coll(pParse, pExpr->pLeft,
>> &unused, &key_info->parts[0].coll_id);
>> diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
>> index 690fa6431..7d1159345 100644
>> --- a/src/box/sql/sqliteInt.h
>> +++ b/src/box/sql/sqliteInt.h
>> @@ -2092,7 +2094,7 @@ struct Expr {
>> u8 op; /* Operation performed by this node */
>> union {
>> /** The affinity of the column. */
>
> 8. Not affinity anymore.
Fixed:
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 216e8d57f..209e7bf0f 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -2093,7 +2093,7 @@ typedef int ynVar;
struct Expr {
u8 op; /* Operation performed by this node */
union {
- /** The affinity of the column. */
+ /** The Type of the column. */
>
>> - enum affinity_type affinity;
>> + enum field_type type;
>> /** Conflict action for RAISE() function. */
>> enum on_conflict_action on_conflict_action;
>> };
>> @@ -4253,26 +4257,30 @@ sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
>> void
>> sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
>> -/**
>> - * Return superposition of two affinities.
>> - * This may be required for determining resulting
>> - * affinity of expressions like a + '2'.
>> - */
>> -enum affinity_type
>> -sql_affinity_result(enum affinity_type aff1, enum affinity_type aff2);
>> +enum field_type
>> +sql_type_result(enum field_type lhs, enum field_type rhs);
>> -int sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity);
>> +enum field_type
>> +sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type);
>> /**
>> - * Return the affinity character for a single column of a table.
>> - * @param def space definition.
>> - * @param idx column index.
>> - * @retval AFFINITY
>> + * Return the type of the expression pExpr.
>> + *
>> + * If pExpr is a column, a reference to a column via an 'AS' alias,
>> + * or a sub-select with a column as the return value, then the
>> + * type of that column is returned. Otherwise, type ANY is returned,
>> + * indicating that the expression can feature any type.
>
> 9. Returning to point 7 - I do not think it should return ANY. Anyway
> you convert it to SCALAR in all places.
See explanation above.
>
>> + *
>> + * The WHERE clause expressions in the following statements all
>> + * have an type:
>> + *
>> + * CREATE TABLE t1(a);
>> + * SELECT * FROM t1 WHERE a;
>> + * SELECT a AS b FROM t1 WHERE b;
>> + * SELECT * FROM t1 WHERE (select a from t1);
>> */
>> -char
>> -sqlite3TableColumnAffinity(struct space_def *def, int idx);
>> -
>> -char sqlite3ExprAffinity(Expr * pExpr);
>> +enum field_type
>> +sql_expr_type(Expr *pExpr);
>> /**
>> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
>> index aae5d6617..06335bcf7 100644
>> --- a/src/box/sql/wherecode.c
>> +++ b/src/box/sql/wherecode.c
>> @@ -392,6 +392,7 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
>> base++;
>> zAff++;
>> }
>> +
>> while (n > 1 && zAff[n - 1] == AFFINITY_BLOB) {
>
> 10. Garbage diff.
Fixed:
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index fe5c56c07..33b860f36 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -392,7 +392,6 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
base++;
zAff++;
}
-
while (n > 1 && zAff[n - 1] == AFFINITY_BLOB) {
n--;
}
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2019-01-16 14:26 ` n.pettik
@ 2019-01-22 15:41 ` Vladislav Shpilevoy
2019-01-28 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-22 15:41 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
>>> -int
>>> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
>>> +enum field_type
>>> +sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type)
>>
>> 4. Strictly speaking, it is not idx_type. It is a type of one
>> column. I think, it should be renamed to just type, or field_type,
>> as well as in struct WhereScan.
>
> Yea, it’s fair. Renamed:
>
> -/*
> - * pExpr is a comparison operator. Return the type affinity that should
> - * be applied to both operands prior to doing the comparison.
> +/**
> + * pExpr is a comparison operator. Return the type affinity
> + * that should be applied to both operands prior to doing
> + * the comparison.
> */
> static enum field_type
> -comparisonAffinity(Expr * pExpr)
> +expr_cmp_mutual_type(struct Expr *pExpr)
> {
> assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
> pExpr->op == TK_GT || pExpr->op == TK_GE || pExpr->op == TK_LE ||
>
Are you sure? Looks like it is not renamed. Also, the comment
was about sql_index_type_is_ok and its parameter idx_type, but you
changed comparisonAffinity function.
>>> + default:
>>> + return 0;
>>> }
>>> }
>>> @@ -2826,16 +2795,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>>> * that columns affinity when building index keys. If <expr> is not
>>> * a column, use numeric affinity.
>>> */
>>> - char affinity; /* Affinity of the LHS of the IN */
>>> int i;
>>> ExprList *pList = pExpr->x.pList;
>>> struct ExprList_item *pItem;
>>> int r1, r2, r3;
>>> - affinity = sqlite3ExprAffinity(pLeft);
>>> - if (!affinity) {
>>> - affinity = AFFINITY_BLOB;
>>> - }
>>> + enum field_type lhs_type =
>>> + sql_expr_type(pLeft);
>>> + if (lhs_type == FIELD_TYPE_ANY)
>>> + lhs_type = FIELD_TYPE_SCALAR;
>>
>> 7. Why did not you move this conversion to sql_expr_type? I mean
>> ANY -> SCALAR.
>
> Unfortunately, right now I can’t replace it with ANY. It is still used
> when IN operator comes with one operand (see EP_Generic flag).
> I have already sent patch which removes it:
> https://github.com/tarantool/tarantool/tree/np/gh-3934-IN-operator-fix
I've already acked the patch. Please, hurry up Kirill Y. to push it,
rebase this branch, and replace ANY with SCALAR.
>
> After it hits 2.1, I will replace SCALAR with ANY.
Vice versa? Not SCALAR with ANY, but ANY with SCALAR, I think.
>
>> ANY differs from SCALAR in only one thing - it is
>> able to store MP_MAP and MP_ARRAY. So I am slightly bent upon
>> frequency of ANY usage in SQL, wherein MAP/ARRAY does not exist.
>
> But still we can SELECT data from spaces created from Lua.
> For instance, _fk_constraint features type ARRAY in its format,
> so we can’t ignore this type even now (IMHO).
Despite our ability to select complex types, in SQL they are all
mere binary data with *sub*type msgpack, not array/map type. So
in SQL we never truly work with anything but SCALAR.
See 5 comments below.
> -/*
> - * Return TRUE if the given expression is a constant which would be
> - * unchanged by OP_ApplyType with the type given in the second
> - * argument.
> - *
> - * This routine is used to determine if the OP_ApplyType operation
> - * can be omitted. When in doubt return FALSE. A false negative
> - * is harmless. A false positive, however, can result in the wrong
> - * answer.
> - */
> int
> -sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
> +sql_expr_needs_no_type_change(const Expr *p, enum field_type type)
1. const Expr -> const struct Expr
> -int sqlite3ExprNeedsNoAffinityChange(const Expr *, char);
> +
> +/**
> + * Return TRUE if the given expression is a constant which would
> + * be unchanged by OP_ApplyType with the type given in the second
> + * argument.
> + *
> + * This routine is used to determine if the OP_ApplyType operation
> + * can be omitted. When in doubt return FALSE. A false negative
> + * is harmless. A false positive, however, can result in the wrong
> + * answer.
> + */
> +int
> +sql_expr_needs_no_type_change(const Expr *epr, enum field_type type);
2. epr -> p, and the same as 1.
> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
> index cd71641b0..d12a2a833 100644
> --- a/src/box/sql/vdbemem.c
> +++ b/src/box/sql/vdbemem.c
> @@ -1316,8 +1315,8 @@ valueFromExpr(sqlite3 * db, /* The database connection */
> sqlite3ValueSetStr(pVal, -1, zVal, SQLITE_DYNAMIC);
> }
> if ((op == TK_INTEGER || op == TK_FLOAT)
> - && affinity == AFFINITY_BLOB) {
> - sqlite3ValueApplyAffinity(pVal, AFFINITY_REAL);
> + && affinity == FIELD_TYPE_SCALAR) {
> + sqlite3ValueApplyAffinity(pVal, FIELD_TYPE_INTEGER);
3. Why before your patch it was REAL and now it is INTEGER?
> } else {
> sqlite3ValueApplyAffinity(pVal, affinity);
> }
> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
> index efbc91cf8..33b860f36 100644
> --- a/src/box/sql/wherecode.c
> +++ b/src/box/sql/wherecode.c
> @@ -423,10 +423,11 @@ updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
> {
> int i;
> for (i = 0; i < n; i++) {
> + enum field_type type = sql_affinity_to_field_type(zAff[i]);
> Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
> - enum affinity_type aff = sqlite3ExprAffinity(p);
> - if (sql_affinity_result(aff, zAff[i]) == AFFINITY_BLOB
> - || sqlite3ExprNeedsNoAffinityChange(p, zAff[i])) {
> + enum field_type expr_type = sql_expr_type(p);
> + if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
> + sql_expr_needs_no_type_change(p, type)) {
> zAff[i] = AFFINITY_BLOB;
> }
> }
4. Something is wrong with formatting. sql_expr_needs_no_type_change should be
aligned under sql_type_result as a part of the condition.
> @@ -1166,20 +1167,12 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
> }
> struct index_def *idx_pk = space->index[0]->def;
> int fieldno = idx_pk->key_def->parts[0].fieldno;
> - char affinity = is_format_set ?
> - space->def->fields[fieldno].affinity :
> - AFFINITY_BLOB;
> - if (affinity == AFFINITY_UNDEFINED) {
> - if (idx_pk->key_def->part_count == 1 &&
> - space->def->fields[fieldno].type ==
> - FIELD_TYPE_INTEGER)
> - affinity = AFFINITY_INTEGER;
> - else
> - affinity = AFFINITY_BLOB;
> - }
> + char fd_type = is_format_set ?
> + space->def->fields[fieldno].type :
> + FIELD_TYPE_ANY;
>
5. Why char? And the alignment is slightly wrong. Non-first lines
should be aligned under is_format_set.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2019-01-22 15:41 ` Vladislav Shpilevoy
@ 2019-01-28 16:39 ` n.pettik
2019-01-30 13:04 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-28 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>>> -int
>>>> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
>>>> +enum field_type
>>>> +sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type)
>>>
>>> 4. Strictly speaking, it is not idx_type. It is a type of one
>>> column. I think, it should be renamed to just type, or field_type,
>>> as well as in struct WhereScan.
>> Yea, it’s fair. Renamed:
>> -/*
>> - * pExpr is a comparison operator. Return the type affinity that should
>> - * be applied to both operands prior to doing the comparison.
>> +/**
>> + * pExpr is a comparison operator. Return the type affinity
>> + * that should be applied to both operands prior to doing
>> + * the comparison.
>> */
>> static enum field_type
>> -comparisonAffinity(Expr * pExpr)
>> +expr_cmp_mutual_type(struct Expr *pExpr)
>> {
>> assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
>> pExpr->op == TK_GT || pExpr->op == TK_GE || pExpr->op == TK_LE ||
>
> Are you sure? Looks like it is not renamed. Also, the comment
> was about sql_index_type_is_ok and its parameter idx_type, but you
> changed comparisonAffinity function.
Sorry, I renamed function, but forgot about param. Fixed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 1de980f00..1d4ea6ef0 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -281,21 +281,21 @@ expr_cmp_mutual_type(struct Expr *pExpr)
/**
* @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
- * @param idx_type is the type of an indexed column.
- * @retval Return true if the index with @idx_type may be used to
+ * @param field_type is the type of an indexed column.
+ * @retval Return true if the index with @field_type may be used to
* implement the comparison in expr.
*/
enum field_type
-sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type)
+sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
{
enum field_type type = expr_cmp_mutual_type(expr);
switch (type) {
case FIELD_TYPE_SCALAR:
return 1;
case FIELD_TYPE_STRING:
- return idx_type == FIELD_TYPE_STRING;
+ return field_type == FIELD_TYPE_STRING;
default:
- return sql_type_is_numeric(idx_type);
+ return sql_type_is_numeric(field_type);
}
}
>>>> @@ -2826,16 +2795,15 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
>>>> * that columns affinity when building index keys. If <expr> is not
>>>> * a column, use numeric affinity.
>>>> */
>>>> - char affinity; /* Affinity of the LHS of the IN */
>>>> int i;
>>>> ExprList *pList = pExpr->x.pList;
>>>> struct ExprList_item *pItem;
>>>> int r1, r2, r3;
>>>> - affinity = sqlite3ExprAffinity(pLeft);
>>>> - if (!affinity) {
>>>> - affinity = AFFINITY_BLOB;
>>>> - }
>>>> + enum field_type lhs_type =
>>>> + sql_expr_type(pLeft);
>>>> + if (lhs_type == FIELD_TYPE_ANY)
>>>> + lhs_type = FIELD_TYPE_SCALAR;
>>>
>>> 7. Why did not you move this conversion to sql_expr_type? I mean
>>> ANY -> SCALAR.
>> Unfortunately, right now I can’t replace it with ANY. It is still used
>> when IN operator comes with one operand (see EP_Generic flag).
>> I have already sent patch which removes it:
>> https://github.com/tarantool/tarantool/tree/np/gh-3934-IN-operator-fix
>
> I've already acked the patch. Please, hurry up Kirill Y. to push it,
> rebase this branch, and replace ANY with SCALAR.
I had to add some other fixes to make this happen:
Changes to sql: replace affinity with field type for func:
diff --git a/src/box/sql/date.c b/src/box/sql/date.c
index e452aad73..f32a5841a 100644
--- a/src/box/sql/date.c
+++ b/src/box/sql/date.c
@@ -1307,14 +1307,14 @@ sqlite3RegisterDateTimeFunctions(void)
static FuncDef aDateTimeFuncs[] = {
#ifndef SQLITE_OMIT_DATETIME_FUNCS
DFUNCTION(julianday, -1, 0, 0, juliandayFunc, FIELD_TYPE_NUMBER),
- DFUNCTION(date, -1, 0, 0, dateFunc, FIELD_TYPE_NUMBER),
- DFUNCTION(time, -1, 0, 0, timeFunc, FIELD_TYPE_NUMBER),
- DFUNCTION(datetime, -1, 0, 0, datetimeFunc, FIELD_TYPE_NUMBER),
- DFUNCTION(strftime, -1, 0, 0, strftimeFunc, FIELD_TYPE_NUMBER),
- DFUNCTION(current_time, 0, 0, 0, ctimeFunc, FIELD_TYPE_NUMBER),
+ DFUNCTION(date, -1, 0, 0, dateFunc, FIELD_TYPE_STRING),
+ DFUNCTION(time, -1, 0, 0, timeFunc, FIELD_TYPE_STRING),
+ DFUNCTION(datetime, -1, 0, 0, datetimeFunc, FIELD_TYPE_STRING),
+ DFUNCTION(strftime, -1, 0, 0, strftimeFunc, FIELD_TYPE_STRING),
+ DFUNCTION(current_time, 0, 0, 0, ctimeFunc, FIELD_TYPE_STRING),
DFUNCTION(current_timestamp, 0, 0, 0, ctimestampFunc,
- FIELD_TYPE_NUMBER),
- DFUNCTION(current_date, 0, 0, 0, cdateFunc, FIELD_TYPE_NUMBER),
+ FIELD_TYPE_STRING),
+ DFUNCTION(current_date, 0, 0, 0, cdateFunc, FIELD_TYPE_STRING),
#else
STR_FUNCTION(current_time, 0, "%H:%M:%S", 0, currentTimeFunc),
STR_FUNCTION(current_date, 0, "%Y-%m-%d", 0, currentTimeFunc),
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 347fee563..58e35e0d9 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3995,7 +3995,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
break;
}
- if (pDef->ret_type != AFFINITY_UNDEFINED) {
+ if (pDef->ret_type != FIELD_TYPE_SCALAR) {
pExpr->affinity =
sql_field_type_to_affinity(pDef->ret_type);
} else {
diff --git a/src/box/sql/func.c b/src/box/sql/func.c
index 08228fbbe..147f93ae4 100644
--- a/src/box/sql/func.c
+++ b/src/box/sql/func.c
@@ -1733,14 +1733,14 @@ sqlite3RegisterBuiltinFunctions(void)
FUNCTION(rtrim, 2, 2, 0, trimFunc, FIELD_TYPE_STRING),
FUNCTION(trim, 1, 3, 0, trimFunc, FIELD_TYPE_STRING),
FUNCTION(trim, 2, 3, 0, trimFunc, FIELD_TYPE_STRING),
- FUNCTION(min, -1, 0, 1, minmaxFunc, 0),
- FUNCTION(min, 0, 0, 1, 0, 0),
+ FUNCTION(min, -1, 0, 1, minmaxFunc, FIELD_TYPE_SCALAR),
+ FUNCTION(min, 0, 0, 1, 0, FIELD_TYPE_SCALAR),
AGGREGATE2(min, 1, 0, 1, minmaxStep, minMaxFinalize,
- SQLITE_FUNC_MINMAX, 0),
- FUNCTION(max, -1, 1, 1, minmaxFunc, 0),
- FUNCTION(max, 0, 1, 1, 0, 0),
+ SQLITE_FUNC_MINMAX, FIELD_TYPE_SCALAR),
+ FUNCTION(max, -1, 1, 1, minmaxFunc, FIELD_TYPE_SCALAR),
+ FUNCTION(max, 0, 1, 1, 0, FIELD_TYPE_SCALAR),
AGGREGATE2(max, 1, 1, 1, minmaxStep, minMaxFinalize,
- SQLITE_FUNC_MINMAX, 0),
+ SQLITE_FUNC_MINMAX, FIELD_TYPE_SCALAR),
FUNCTION2(typeof, 1, 0, 0, typeofFunc, SQLITE_FUNC_TYPEOF,
FIELD_TYPE_STRING),
FUNCTION2(length, 1, 0, 0, lengthFunc, SQLITE_FUNC_LENGTH,
@@ -1769,9 +1769,9 @@ sqlite3RegisterBuiltinFunctions(void)
FUNCTION(zeroblob, 1, 0, 0, zeroblobFunc, FIELD_TYPE_SCALAR),
FUNCTION(substr, 2, 0, 0, substrFunc, FIELD_TYPE_STRING),
FUNCTION(substr, 3, 0, 0, substrFunc, FIELD_TYPE_STRING),
- AGGREGATE(sum, 1, 0, 0, sumStep, sumFinalize, 0),
- AGGREGATE(total, 1, 0, 0, sumStep, totalFinalize, 0),
- AGGREGATE(avg, 1, 0, 0, sumStep, avgFinalize, 0),
+ AGGREGATE(sum, 1, 0, 0, sumStep, sumFinalize, FIELD_TYPE_NUMBER),
+ AGGREGATE(total, 1, 0, 0, sumStep, totalFinalize, FIELD_TYPE_NUMBER),
+ AGGREGATE(avg, 1, 0, 0, sumStep, avgFinalize, FIELD_TYPE_NUMBER),
AGGREGATE2(count, 0, 0, 0, countStep, countFinalize,
SQLITE_FUNC_COUNT, FIELD_TYPE_INTEGER),
AGGREGATE(count, 1, 0, 0, countStep, countFinalize,
@@ -1788,9 +1788,10 @@ sqlite3RegisterBuiltinFunctions(void)
#ifdef SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
FUNCTION(unknown, -1, 0, 0, unknownFunc, 0),
#endif
- FUNCTION(coalesce, 1, 0, 0, 0, 0),
- FUNCTION(coalesce, 0, 0, 0, 0, 0),
- FUNCTION2(coalesce, -1, 0, 0, noopFunc, SQLITE_FUNC_COALESCE, 0),
+ FUNCTION(coalesce, 1, 0, 0, 0, FIELD_TYPE_SCALAR),
+ FUNCTION(coalesce, 0, 0, 0, 0, FIELD_TYPE_SCALAR),
+ FUNCTION2(coalesce, -1, 0, 0, noopFunc, SQLITE_FUNC_COALESCE,
+ FIELD_TYPE_SCALAR),
};
sqlite3AnalyzeFunctions();
sqlite3RegisterDateTimeFunctions();
diff --git a/src/box/sql/resolve.c b/src/box/sql/resolve.c
index 4c6baaac6..cbda48f78 100644
--- a/src/box/sql/resolve.c
+++ b/src/box/sql/resolve.c
@@ -637,6 +637,8 @@ resolveExprStep(Walker * pWalker, Expr * pExpr)
}
} else {
is_agg = pDef->xFinalize != 0;
+ pExpr->affinity =
+ sql_field_type_to_affinity(pDef->ret_type);
if (pDef->funcFlags & SQLITE_FUNC_UNLIKELY) {
ExprSetProperty(pExpr,
EP_Unlikely | EP_Skip);
diff --git a/test-run b/test-run
index 02207efd2..4c3f11812 160000
--- a/test-run
+++ b/test-run
@@ -1 +1 @@
-Subproject commit 02207efd2ca44067b76c79bf012f142016d929ae
+Subproject commit 4c3f11812c0c17f08780f31b6a748eef1126b2e2
diff --git a/test/sql-tap/analyze9.test.lua b/test/sql-tap/analyze9.test.lua
index df62a1624..c43437712 100755
--- a/test/sql-tap/analyze9.test.lua
+++ b/test/sql-tap/analyze9.test.lua
@@ -58,6 +58,9 @@ msgpack_decode_sample = function(txt)
end
i = i+1
end
+ if type(decoded_str) == "number" then
+ return tostring(decoded_str)
+ end
return decoded_str
end
@@ -332,7 +335,7 @@ test:do_execsql_test(
test:do_execsql_test(
4.7,
[[
- SELECT count(*) FROM "_sql_stat4" WHERE msgpack_decode_sample("sample") IN (34, 68, 102, 136, 170, 204, 238, 272);
+ SELECT count(*) FROM "_sql_stat4" WHERE lrange(msgpack_decode_sample("sample"), 1, 1) IN ('34', '68', '102', '136', '170', '204', '238', '272');
]], {
-- <4.7>
8
@@ -367,8 +370,8 @@ test:do_execsql_test(
SELECT msgpack_decode_sample("sample") FROM "_sql_stat4";
]], {
-- <4.9>
- "x", 1110, 2230, 2750, 3350, 4090, 4470, 4980, 5240, 5280, 5290, 5590, 5920,
- 5930, 6220, 6710, 7000, 7710, 7830, 7970, 8890, 8950, 9240, 9250, 9680
+ "x", "1110", "2230", "2750", "3350", "4090", "4470", "4980", "5240", "5280", "5290", "5590", "5920",
+ "5930", "6220", "6710", "7000", "7710", "7830", "7970", "8890", "8950", "9240", "9250", "9680"
-- </4.9>
})
Additional changes to sql: replace affinity with field type in struct Expr:
Firstly, I placed several asserts like assert(type != FIELD_TYPE_ANY);
and noted that not for all expressions its true. So I patched sql_expr_type
function:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index fdf00273a..e1a0dfa73 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -74,10 +74,29 @@ sql_expr_type(struct Expr *pExpr)
case TK_MINUS:
case TK_STAR:
case TK_SLASH:
+ case TK_REM:
+ case TK_BITAND:
+ case TK_BITOR:
+ case TK_LSHIFT:
+ case TK_RSHIFT:
assert(pExpr->pRight != NULL && pExpr->pLeft != NULL);
enum field_type lhs_type = sql_expr_type(pExpr->pLeft);
enum field_type rhs_type = sql_expr_type(pExpr->pRight);
return sql_type_result(rhs_type, lhs_type);
+ case TK_CONCAT:
+ return FIELD_TYPE_STRING;
+ case TK_CASE:
+ assert(pExpr->x.pList->nExpr >= 2);
+ /*
+ * CASE expression comes at least with one
+ * WHEN and one THEN clauses. So, first
+ * expression always represents WHEN
+ * argument, and the second one - THEN.
+ *
+ * TODO: We must ensure that all THEN clauses
+ * have arguments of the same type.
+ */
+ return sql_expr_type(pExpr->x.pList->a[1].pExpr);
case TK_LT:
case TK_GT:
case TK_EQ:
@@ -92,16 +111,14 @@ sql_expr_type(struct Expr *pExpr)
* return INTEGER.
*/
return FIELD_TYPE_INTEGER;
- }
- /*
- * In case of unary plus we shouldn't discard
- * type of operand (since plus always features
- * NUMERIC type).
- */
- if (op == TK_UPLUS) {
+ case TK_UMINUS:
+ case TK_UPLUS:
+ case TK_NO:
+ case TK_BITNOT:
assert(pExpr->pRight == NULL);
- return pExpr->pLeft->type;
+ return sql_expr_type(pExpr->pLeft);
}
+
return pExpr->type;
}
Then, I make default type of expression be SCALAR
except for binding params:
diff --git a/src/box/sql/parse.y b/src/box/sql/parse.y
index ef691cb5c..d6fc41e3b 100644
--- a/src/box/sql/parse.y
+++ b/src/box/sql/parse.y
@@ -829,6 +829,18 @@ idlist(A) ::= nm(Y).
case TK_FLOAT:
p->type = FIELD_TYPE_NUMBER;
break;
+ case TK_VARIABLE:
+ /*
+ * For variables we set BOOLEAN type since
+ * unassigned bindings will be replaced
+ * with NULL automatically, i.e. without
+ * explicit call of sql_bind_value().
+ */
+ p->type = FIELD_TYPE_BOOLEAN;
+ break;
+ default:
+ p->type = FIELD_TYPE_SCALAR;
+ break;
}
p->op = (u8)op;
p->flags = EP_Leaf;
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 8a8ccc68c..2cc0a4a78 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1725,6 +1726,8 @@ generateColumnNames(Parse * pParse, /* Parser context */
p = pEList->a[i].pExpr;
if (NEVER(p == 0))
continue;
+ if (p->op == TK_VARIABLE)
+ var_pos[var_count++] = i;
switch (p->type) {
case FIELD_TYPE_INTEGER:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "INTEGER",
@@ -1742,21 +1745,12 @@ generateColumnNames(Parse * pParse, /* Parser context */
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BLOB",
SQLITE_TRANSIENT);
break;
- default: ;
- char *type;
- /*
- * For variables we set BOOLEAN type since
- * unassigned bindings will be replaced
- * with NULL automatically, i.e. without
- * explicit call of sql_bind_value().
- */
- if (p->op == TK_VARIABLE) {
- var_pos[var_count++] = i;
- type = "BOOLEAN";
- } else {
- type = "UNKNOWN";
- }
- sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, type,
+ case FIELD_TYPE_BOOLEAN:
+ sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BOOLEAN",
+ SQLITE_TRANSIENT);
+ break;
+ default:
+ sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "UNKNOWN",
SQLITE_TRANSIENT);
}
if (pEList->a[i].zName) {
Finally, I simplified derived type calculations and removed
all ANY -> SCALAR transformations:
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 8a8ccc68c..2cc0a4a78 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1973,8 +1967,6 @@ sqlite3SelectAddColumnTypeAndCollation(Parse * pParse, /* Parsing contexts */
for (uint32_t i = 0; i < pTab->def->field_count; i++) {
p = a[i].pExpr;
enum field_type type = sql_expr_type(p);
- if (type == FIELD_TYPE_ANY)
- type = FIELD_TYPE_SCALAR;
pTab->def->fields[i].affinity = sql_field_type_to_affinity(type);
pTab->def->fields[i].type = type;
bool is_found;
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index 930b53b2b..44305f7dc 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -1169,7 +1169,7 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
int fieldno = idx_pk->key_def->parts[0].fieldno;
enum field_type fd_type = is_format_set ?
space->def->fields[fieldno].type :
- FIELD_TYPE_ANY;
+ FIELD_TYPE_SCALAR;
uint32_t pk_part_count = idx_pk->key_def->part_count;
if (pk_part_count == 1 && fd_type == FIELD_TYPE_INTEGER) {
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 6ffd5ae84..1fb5fa593 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -758,7 +758,7 @@ exprAnalyzeOrTerm(SrcList * pSrc, /* the FROM clause */
enum field_type lhs =
sql_expr_type(pOrTerm->pExpr->
pLeft);
- if (rhs != FIELD_TYPE_ANY &&
+ if (rhs != FIELD_TYPE_SCALAR &&
rhs != lhs) {
okToChngToIN = 0;
} else {
diff --git a/src/box/sql.c b/src/box/sql.c
index 530ec2384..cb8f6a60a 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -387,8 +387,8 @@ sql_ephemeral_space_create(uint32_t field_count, struct sql_key_info *key_info)
part->sort_order = SORT_ORDER_ASC;
if (def != NULL && i < def->part_count) {
assert(def->parts[i].type < field_type_MAX);
- part->type = def->parts[i].type != FIELD_TYPE_ANY ?
- def->parts[i].type : FIELD_TYPE_SCALAR;
+ assert(def->parts[i].type != FIELD_TYPE_ANY);
+ part->type = def->parts[i].type;
part->coll_id = def->parts[i].coll_id;
} else {
part->coll_id = COLL_NONE;
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index fdf00273a..e1a0dfa73 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -231,22 +248,9 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
enum field_type
sql_type_result(enum field_type lhs, enum field_type rhs)
{
- if (lhs != FIELD_TYPE_ANY && rhs != FIELD_TYPE_ANY) {
- if (sql_type_is_numeric(lhs) || sql_type_is_numeric(rhs))
- return FIELD_TYPE_NUMBER;
- else
- return FIELD_TYPE_SCALAR;
- } else if (lhs == FIELD_TYPE_ANY && rhs == FIELD_TYPE_ANY) {
- /*
- * Neither side of the comparison is a column.
- * Compare the results directly.
- */
- return FIELD_TYPE_SCALAR;
- } else {
- if (lhs == FIELD_TYPE_ANY)
- return rhs;
- return lhs;
- }
+ if (sql_type_is_numeric(lhs) || sql_type_is_numeric(rhs))
+ return FIELD_TYPE_NUMBER;
+ return FIELD_TYPE_SCALAR;
}
@@ -2784,8 +2788,6 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
enum field_type lhs_type =
sql_expr_type(pLeft);
- if (lhs_type == FIELD_TYPE_ANY)
- lhs_type = FIELD_TYPE_SCALAR;
bool unused;
sql_expr_coll(pParse, pExpr->pLeft,
&unused, &key_info->parts[0].coll_id);
Test below are changed due to correct calculation of derived
type for unary operators: you may notice that before it was
simple expr->type. However, it didn’t give us original type
of expr: for column, for instance we should fetch it from space_def.
diff --git a/test/sql-tap/where2.test.lua b/test/sql-tap/where2.test.lua
index 9089c97f9..56bec153f 100755
--- a/test/sql-tap/where2.test.lua
+++ b/test/sql-tap/where2.test.lua
@@ -653,14 +653,13 @@ test:do_test(
"where2-6.9",
function()
return queryplan([[
- -- The + operator removes affinity from the rhs. No conversions
- -- occur and the comparison is false. The result is an empty set.
+ -- The + operator doesn't affect RHS.
--
SELECT b,a FROM t2249b CROSS JOIN t2249a WHERE a=+b;
]])
end, {
-- <where2-6.9>
- "nosort", "T2249B", "*", "T2249A", "*"
+ 123, "0123", "nosort", "T2249B", "*", "T2249A", "*"
-- </where2-6.9>
})
@@ -673,7 +672,7 @@ test:do_test(
]])
end, {
-- <where2-6.9.2>
- "nosort", "T2249B", "*", "T2249A", "*"
+ 123, "0123","nosort", "T2249B", "*", "T2249A", "*"
-- </where2-6.9.2>
})
@@ -681,9 +680,6 @@ test:do_test(
"where2-6.10",
function()
return queryplan([[
- -- Use + on both sides of the comparison to disable indices
- -- completely. Make sure we get the same result.
- --
SELECT b,a FROM t2249b CROSS JOIN t2249a WHERE +a=+b;
]])
end, {
@@ -753,45 +749,36 @@ test:do_test(
test:do_test(
"where2-6.12",
function()
- -- In this case, the +b disables the affinity conflict and allows
- -- the OR optimization to be used again. The result is now an empty
- -- set, the same as in where2-6.9.
return queryplan([[
SELECT b,a FROM t2249b CROSS JOIN t2249a WHERE a=+b OR a='hello';
]])
end, {
-- <where2-6.12>
- "nosort", "T2249B", "*", "T2249A", "*"
+ 123, "0123", "nosort", "T2249B", "*", "T2249A", "*"
-- </where2-6.12>
})
test:do_test(
"where2-6.12.2",
function()
- -- In this case, the +b disables the affinity conflict and allows
- -- the OR optimization to be used again. The result is now an empty
- -- set, the same as in where2-6.9.
return queryplan([[
SELECT b,a FROM t2249b CROSS JOIN t2249a WHERE a='hello' OR +b=a;
]])
end, {
-- <where2-6.12.2>
- "nosort", "T2249B", "*", "T2249A", "*"
+ 123, "0123", "nosort", "T2249B", "*", "T2249A", "*"
-- </where2-6.12.2>
})
test:do_test(
"where2-6.12.3",
function()
- -- In this case, the +b disables the affinity conflict and allows
- -- the OR optimization to be used again. The result is now an empty
- -- set, the same as in where2-6.9.
return queryplan([[
SELECT b,a FROM t2249b CROSS JOIN t2249a WHERE +b=a OR a='hello';
]])
end, {
-- <where2-6.12.3>
- "nosort", "T2249B", "*", "T2249A", "*"
+ 123, "0123", "nosort", "T2249B", "*", "T2249A", "*"
-- </where2-6.12.3>
})
>
>> After it hits 2.1, I will replace SCALAR with ANY.
>
> Vice versa? Not SCALAR with ANY, but ANY with SCALAR, I think.
Yep.
>>> ANY differs from SCALAR in only one thing - it is
>>> able to store MP_MAP and MP_ARRAY. So I am slightly bent upon
>>> frequency of ANY usage in SQL, wherein MAP/ARRAY does not exist.
>> But still we can SELECT data from spaces created from Lua.
>> For instance, _fk_constraint features type ARRAY in its format,
>> so we can’t ignore this type even now (IMHO).
>
> Despite our ability to select complex types, in SQL they are all
> mere binary data with *sub*type msgpack, not array/map type. So
> in SQL we never truly work with anything but SCALAR.
Wait, at the stage of query compiling we don’t operate on
any subtypes. We simply fetch space_def->fields->type,
which may turn out to be FIELD_TYPE_MAP/ARRAY or
whatever.
>> -/*
>> - * Return TRUE if the given expression is a constant which would be
>> - * unchanged by OP_ApplyType with the type given in the second
>> - * argument.
>> - *
>> - * This routine is used to determine if the OP_ApplyType operation
>> - * can be omitted. When in doubt return FALSE. A false negative
>> - * is harmless. A false positive, however, can result in the wrong
>> - * answer.
>> - */
>> int
>> -sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
>> +sql_expr_needs_no_type_change(const Expr *p, enum field_type type)
>
> 1. const Expr -> const struct Expr
Fixed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 1de980f00..ddce8c13c 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -2117,7 +2117,7 @@ sqlite3ExprCanBeNull(const Expr * p)
}
int
-sql_expr_needs_no_type_change(const Expr *p, enum field_type type)
+sql_expr_needs_no_type_change(const struct Expr *p, enum field_type type)
{
u8 op;
if (type == FIELD_TYPE_SCALAR)
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index f44235cb6..fac650f62 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3721,7 +3721,7 @@ int sqlite3ExprCanBeNull(const Expr *);
* answer.
*/
int
-sql_expr_needs_no_type_change(const Expr *epr, enum field_type type);
+sql_expr_needs_no_type_change(const struct Expr *expr, enum field_type type);
>> +/**
>> + * Return TRUE if the given expression is a constant which would
>> + * be unchanged by OP_ApplyType with the type given in the second
>> + * argument.
>> + *
>> + * This routine is used to determine if the OP_ApplyType operation
>> + * can be omitted. When in doubt return FALSE. A false negative
>> + * is harmless. A false positive, however, can result in the wrong
>> + * answer.
>> + */
>> +int
>> +sql_expr_needs_no_type_change(const Expr *epr, enum field_type type);
>
> 2. epr -> p, and the same as 1.
Fixed, see above.
>> diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
>> index cd71641b0..d12a2a833 100644
>> --- a/src/box/sql/vdbemem.c
>> +++ b/src/box/sql/vdbemem.c
>> @@ -1316,8 +1315,8 @@ valueFromExpr(sqlite3 * db, /* The database connection */
>> sqlite3ValueSetStr(pVal, -1, zVal, SQLITE_DYNAMIC);
>> }
>> if ((op == TK_INTEGER || op == TK_FLOAT)
>> - && affinity == AFFINITY_BLOB) {
>> - sqlite3ValueApplyAffinity(pVal, AFFINITY_REAL);
>> + && affinity == FIELD_TYPE_SCALAR) {
>> + sqlite3ValueApplyAffinity(pVal, FIELD_TYPE_INTEGER);
>
> 3. Why before your patch it was REAL and now it is INTEGER?
Typo. Fixed in previous patch (while I was renaming affinity vars to type).
>> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
>> index efbc91cf8..33b860f36 100644
>> --- a/src/box/sql/wherecode.c
>> +++ b/src/box/sql/wherecode.c
>> @@ -423,10 +423,11 @@ updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
>> {
>> int i;
>> for (i = 0; i < n; i++) {
>> + enum field_type type = sql_affinity_to_field_type(zAff[i]);
>> Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
>> - enum affinity_type aff = sqlite3ExprAffinity(p);
>> - if (sql_affinity_result(aff, zAff[i]) == AFFINITY_BLOB
>> - || sqlite3ExprNeedsNoAffinityChange(p, zAff[i])) {
>> + enum field_type expr_type = sql_expr_type(p);
>> + if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
>> + sql_expr_needs_no_type_change(p, type)) {
>> zAff[i] = AFFINITY_BLOB;
>> }
>> }
>
> 4. Something is wrong with formatting. sql_expr_needs_no_type_change should be
> aligned under sql_type_result as a part of the condition.
Fixed:
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index 33b860f36..bc57efe18 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -427,7 +427,7 @@ updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
enum field_type expr_type = sql_expr_type(p);
if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
- sql_expr_needs_no_type_change(p, type)) {
+ sql_expr_needs_no_type_change(p, type)) {
zAff[i] = AFFINITY_BLOB;
}
}
>> @@ -1166,20 +1167,12 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
>> }
>> struct index_def *idx_pk = space->index[0]->def;
>> int fieldno = idx_pk->key_def->parts[0].fieldno;
>> - char affinity = is_format_set ?
>> - space->def->fields[fieldno].affinity :
>> - AFFINITY_BLOB;
>> - if (affinity == AFFINITY_UNDEFINED) {
>> - if (idx_pk->key_def->part_count == 1 &&
>> - space->def->fields[fieldno].type ==
>> - FIELD_TYPE_INTEGER)
>> - affinity = AFFINITY_INTEGER;
>> - else
>> - affinity = AFFINITY_BLOB;
>> - }
>> + char fd_type = is_format_set ?
>> + space->def->fields[fieldno].type :
>> + FIELD_TYPE_ANY;
>>
>
> 5. Why char? And the alignment is slightly wrong. Non-first lines
> should be aligned under is_format_set.
Fixed:
@@ -1167,9 +1167,9 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
}
struct index_def *idx_pk = space->index[0]->def;
int fieldno = idx_pk->key_def->parts[0].fieldno;
- char fd_type = is_format_set ?
- space->def->fields[fieldno].type :
- FIELD_TYPE_ANY;
+ enum field_type fd_type = is_format_set ?
+ space->def->fields[fieldno].type :
+ FIELD_TYPE_SCALAR;
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2019-01-28 16:39 ` n.pettik
@ 2019-01-30 13:04 ` Vladislav Shpilevoy
2019-02-01 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-30 13:04 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
>>>> 7. Why did not you move this conversion to sql_expr_type? I mean
>>>> ANY -> SCALAR.
>>> Unfortunately, right now I can’t replace it with ANY. It is still used
>>> when IN operator comes with one operand (see EP_Generic flag).
>>> I have already sent patch which removes it:
>>> https://github.com/tarantool/tarantool/tree/np/gh-3934-IN-operator-fix
>>
>> I've already acked the patch. Please, hurry up Kirill Y. to push it,
>> rebase this branch, and replace ANY with SCALAR.
>
> I had to add some other fixes to make this happen:
>
> Changes to sql: replace affinity with field type for func:
>
> diff --git a/test-run b/test-run
> index 02207efd2..4c3f11812 160000
> --- a/test-run
> +++ b/test-run
> @@ -1 +1 @@
> -Subproject commit 02207efd2ca44067b76c79bf012f142016d929ae
> +Subproject commit 4c3f11812c0c17f08780f31b6a748eef1126b2e2
1. ???
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index fdf00273a..e1a0dfa73 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -92,16 +111,14 @@ sql_expr_type(struct Expr *pExpr)
> * return INTEGER.
> */
> return FIELD_TYPE_INTEGER;
> - }
> - /*
> - * In case of unary plus we shouldn't discard
> - * type of operand (since plus always features
> - * NUMERIC type).
> - */
> - if (op == TK_UPLUS) {
> + case TK_UMINUS:
> + case TK_UPLUS:
> + case TK_NO:
> + case TK_BITNOT:
> assert(pExpr->pRight == NULL);
> - return pExpr->pLeft->type;
> + return sql_expr_type(pExpr->pLeft);
> }
> +
2. Stray empty line.
> return pExpr->type;
>>>> ANY differs from SCALAR in only one thing - it is
>>>> able to store MP_MAP and MP_ARRAY. So I am slightly bent upon
>>>> frequency of ANY usage in SQL, wherein MAP/ARRAY does not exist.
>>> But still we can SELECT data from spaces created from Lua.
>>> For instance, _fk_constraint features type ARRAY in its format,
>>> so we can’t ignore this type even now (IMHO).
>>
>> Despite our ability to select complex types, in SQL they are all
>> mere binary data with *sub*type msgpack, not array/map type. So
>> in SQL we never truly work with anything but SCALAR.
>
> Wait, at the stage of query compiling we don’t operate on
> any subtypes. We simply fetch space_def->fields->type,
> which may turn out to be FIELD_TYPE_MAP/ARRAY or
> whatever.
Hmm, yes, you are right. We should consider array and map too.
See 4 comments below.
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index d263c1bb5..e1a0dfa73 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -81,27 +57,46 @@ sqlite3ExprAffinity(Expr * pExpr)
> case TK_SELECT:
> assert(pExpr->flags & EP_xIsSelect);
> el = pExpr->x.pSelect->pEList;
> - return sqlite3ExprAffinity(el->a[0].pExpr);
> + return sql_expr_type(el->a[0].pExpr);
> case TK_CAST:
> assert(!ExprHasProperty(pExpr, EP_IntValue));
> - return pExpr->affinity;
> + return pExpr->type;
> case TK_AGG_COLUMN:
> case TK_COLUMN:
> + case TK_TRIGGER:
> assert(pExpr->iColumn >= 0);
> - return sqlite3TableColumnAffinity(pExpr->space_def,
> - pExpr->iColumn);
> + return pExpr->space_def->fields[pExpr->iColumn].type;
> case TK_SELECT_COLUMN:
> assert(pExpr->pLeft->flags & EP_xIsSelect);
> el = pExpr->pLeft->x.pSelect->pEList;
> - return sqlite3ExprAffinity(el->a[pExpr->iColumn].pExpr);
> + return sql_expr_type(el->a[pExpr->iColumn].pExpr);
> case TK_PLUS:
> case TK_MINUS:
> case TK_STAR:
> case TK_SLASH:
> + case TK_REM:
> + case TK_BITAND:
> + case TK_BITOR:
> + case TK_LSHIFT:
> + case TK_RSHIFT:
> assert(pExpr->pRight != NULL && pExpr->pLeft != NULL);
> - enum affinity_type lhs_aff = sqlite3ExprAffinity(pExpr->pLeft);
> - enum affinity_type rhs_aff = sqlite3ExprAffinity(pExpr->pRight);
> - return sql_affinity_result(rhs_aff, lhs_aff);
> + enum field_type lhs_type = sql_expr_type(pExpr->pLeft);
> + enum field_type rhs_type = sql_expr_type(pExpr->pRight);
> + return sql_type_result(rhs_type, lhs_type);
> + case TK_CONCAT:
> + return FIELD_TYPE_STRING;
1. Indentation.
> + case TK_CASE:
> + assert(pExpr->x.pList->nExpr >= 2);
> + /*
> + * CASE expression comes at least with one
> + * WHEN and one THEN clauses. So, first
> + * expression always represents WHEN
> + * argument, and the second one - THEN.
> + *
> + * TODO: We must ensure that all THEN clauses
> + * have arguments of the same type.
> + */
> + return sql_expr_type(pExpr->x.pList->a[1].pExpr);
> case TK_LT:
> case TK_GT:
> case TK_EQ:
> @@ -252,74 +245,57 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
> -/*
> - * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
> - * idx_affinity is the affinity of an indexed column. Return true
> - * if the index with affinity idx_affinity may be used to implement
> - * the comparison in pExpr.
> +/**
> + * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
> + * @param field_type is the type of an indexed column.
> + * @retval Return true if the index with @field_type may be used to
> + * implement the comparison in expr.
> */
> -int
> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
> +enum field_type
> +sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
2. Returns 'bool true', but the return value type is enum field_type?
Please, change to bool.
> {
> - char aff = comparisonAffinity(pExpr);
> - switch (aff) {
> - case AFFINITY_BLOB:
> + enum field_type type = expr_cmp_mutual_type(expr);
> + switch (type) {
> + case FIELD_TYPE_SCALAR:
> return 1;
3. 1 -> true
> - case AFFINITY_TEXT:
> - return idx_affinity == AFFINITY_TEXT;
> + case FIELD_TYPE_STRING:
> + return field_type == FIELD_TYPE_STRING;
> default:
> - return sqlite3IsNumericAffinity(idx_affinity);
> + return sql_type_is_numeric(field_type);
> }
> }
>
> @@ -2140,21 +2116,11 @@ sqlite3ExprCanBeNull(const Expr * p)
> }
> }
>
> -/*
> - * Return TRUE if the given expression is a constant which would be
> - * unchanged by OP_ApplyType with the type given in the second
> - * argument.
> - *
> - * This routine is used to determine if the OP_ApplyType operation
> - * can be omitted. When in doubt return FALSE. A false negative
> - * is harmless. A false positive, however, can result in the wrong
> - * answer.
> - */
> int
> -sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
> +sql_expr_needs_no_type_change(const struct Expr *p, enum field_type type)
4. The same. Anyway you changed the function name, so without
increasing the diff you can change its return value to bool,
and the values below 0 -> false, 1 -> true.
> {
> u8 op;
> - if (aff == AFFINITY_BLOB)
> + if (type == FIELD_TYPE_SCALAR)
> return 1;
> while (p->op == TK_UPLUS || p->op == TK_UMINUS) {
> p = p->pLeft;
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2019-01-30 13:04 ` Vladislav Shpilevoy
@ 2019-02-01 16:39 ` n.pettik
2019-02-05 15:08 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-02-01 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>>>> 7. Why did not you move this conversion to sql_expr_type? I mean
>>>>> ANY -> SCALAR.
>>>> Unfortunately, right now I can’t replace it with ANY. It is still used
>>>> when IN operator comes with one operand (see EP_Generic flag).
>>>> I have already sent patch which removes it:
>>>> https://github.com/tarantool/tarantool/tree/np/gh-3934-IN-operator-fix
>>>
>>> I've already acked the patch. Please, hurry up Kirill Y. to push it,
>>> rebase this branch, and replace ANY with SCALAR.
>> I had to add some other fixes to make this happen:
>> Changes to sql: replace affinity with field type for func:
>> diff --git a/test-run b/test-run
>> index 02207efd2..4c3f11812 160000
>> --- a/test-run
>> +++ b/test-run
>> @@ -1 +1 @@
>> -Subproject commit 02207efd2ca44067b76c79bf012f142016d929ae
>> +Subproject commit 4c3f11812c0c17f08780f31b6a748eef1126b2e2
>
> 1. ???
Accidentally trapped to diff.
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index fdf00273a..e1a0dfa73 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -92,16 +111,14 @@ sql_expr_type(struct Expr *pExpr)
>> * return INTEGER.
>> */
>> return FIELD_TYPE_INTEGER;
>> - }
>> - /*
>> - * In case of unary plus we shouldn't discard
>> - * type of operand (since plus always features
>> - * NUMERIC type).
>> - */
>> - if (op == TK_UPLUS) {
>> + case TK_UMINUS:
>> + case TK_UPLUS:
>> + case TK_NO:
>> + case TK_BITNOT:
>> assert(pExpr->pRight == NULL);
>> - return pExpr->pLeft->type;
>> + return sql_expr_type(pExpr->pLeft);
>> }
>> +
>
> 2. Stray empty line.
Removed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index e1a0dfa73..4ff7169ab 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -118,7 +118,6 @@ sql_expr_type(struct Expr *pExpr)
assert(pExpr->pRight == NULL);
return sql_expr_type(pExpr->pLeft);
}
-
return pExpr->type;
}
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index d263c1bb5..e1a0dfa73 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -81,27 +57,46 @@ sqlite3ExprAffinity(Expr * pExpr)
>> case TK_SELECT:
>> assert(pExpr->flags & EP_xIsSelect);
>> el = pExpr->x.pSelect->pEList;
>> - return sqlite3ExprAffinity(el->a[0].pExpr);
>> + return sql_expr_type(el->a[0].pExpr);
>> case TK_CAST:
>> assert(!ExprHasProperty(pExpr, EP_IntValue));
>> - return pExpr->affinity;
>> + return pExpr->type;
>> case TK_AGG_COLUMN:
>> case TK_COLUMN:
>> + case TK_TRIGGER:
>> assert(pExpr->iColumn >= 0);
>> - return sqlite3TableColumnAffinity(pExpr->space_def,
>> - pExpr->iColumn);
>> + return pExpr->space_def->fields[pExpr->iColumn].type;
>> case TK_SELECT_COLUMN:
>> assert(pExpr->pLeft->flags & EP_xIsSelect);
>> el = pExpr->pLeft->x.pSelect->pEList;
>> - return sqlite3ExprAffinity(el->a[pExpr->iColumn].pExpr);
>> + return sql_expr_type(el->a[pExpr->iColumn].pExpr);
>> case TK_PLUS:
>> case TK_MINUS:
>> case TK_STAR:
>> case TK_SLASH:
>> + case TK_REM:
>> + case TK_BITAND:
>> + case TK_BITOR:
>> + case TK_LSHIFT:
>> + case TK_RSHIFT:
>> assert(pExpr->pRight != NULL && pExpr->pLeft != NULL);
>> - enum affinity_type lhs_aff = sqlite3ExprAffinity(pExpr->pLeft);
>> - enum affinity_type rhs_aff = sqlite3ExprAffinity(pExpr->pRight);
>> - return sql_affinity_result(rhs_aff, lhs_aff);
>> + enum field_type lhs_type = sql_expr_type(pExpr->pLeft);
>> + enum field_type rhs_type = sql_expr_type(pExpr->pRight);
>> + return sql_type_result(rhs_type, lhs_type);
>> + case TK_CONCAT:
>> + return FIELD_TYPE_STRING;
>
> 1. Indentation.
Fixed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index e1a0dfa73..91d61b2d9 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -84,7 +84,7 @@ sql_expr_type(struct Expr *pExpr)
enum field_type rhs_type = sql_expr_type(pExpr->pRight);
return sql_type_result(rhs_type, lhs_type);
case TK_CONCAT:
- return FIELD_TYPE_STRING;
+ return FIELD_TYPE_STRING;
case TK_CASE:
>> @@ -252,74 +245,57 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
>> -/*
>> - * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
>> - * idx_affinity is the affinity of an indexed column. Return true
>> - * if the index with affinity idx_affinity may be used to implement
>> - * the comparison in pExpr.
>> +/**
>> + * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
>> + * @param field_type is the type of an indexed column.
>> + * @retval Return true if the index with @field_type may be used to
>> + * implement the comparison in expr.
>> */
>> -int
>> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
>> +enum field_type
>> +sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
>
> 2. Returns 'bool true', but the return value type is enum field_type?
> Please, change to bool.
Fixed:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index e1a0dfa73..2bc4f00cf 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -285,7 +284,7 @@ expr_cmp_mutual_type(struct Expr *pExpr)
* @retval Return true if the index with @field_type may be used to
* implement the comparison in expr.
*/
-enum field_type
+bool
sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
{
enum field_type type = expr_cmp_mutual_type(expr);
switch (type) {
case FIELD_TYPE_SCALAR:
- return 1;
+ return true;
case FIELD_TYPE_STRING:
return field_type == FIELD_TYPE_STRING;
default:
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index d892631d8..8f75546de 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -4284,7 +4284,7 @@ sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
enum field_type
sql_type_result(enum field_type lhs, enum field_type rhs);
-enum field_type
+bool
sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type);
>> {
>> - char aff = comparisonAffinity(pExpr);
>> - switch (aff) {
>> - case AFFINITY_BLOB:
>> + enum field_type type = expr_cmp_mutual_type(expr);
>> + switch (type) {
>> + case FIELD_TYPE_SCALAR:
>> return 1;
>
> 3. 1 -> true
Fixed, see above.
>> @@ -2140,21 +2116,11 @@ sqlite3ExprCanBeNull(const Expr * p)
>> }
>> }
>> -/*
>> - * Return TRUE if the given expression is a constant which would be
>> - * unchanged by OP_ApplyType with the type given in the second
>> - * argument.
>> - *
>> - * This routine is used to determine if the OP_ApplyType operation
>> - * can be omitted. When in doubt return FALSE. A false negative
>> - * is harmless. A false positive, however, can result in the wrong
>> - * answer.
>> - */
>> int
>> -sqlite3ExprNeedsNoAffinityChange(const Expr * p, char aff)
>> +sql_expr_needs_no_type_change(const struct Expr *p, enum field_type type)
>
> 4. The same. Anyway you changed the function name, so without
> increasing the diff you can change its return value to bool,
> and the values below 0 -> false, 1 -> true.
That’s true. Fixed:
@@ -2116,12 +2115,12 @@ sqlite3ExprCanBeNull(const Expr * p)
}
}
-int
+bool
sql_expr_needs_no_type_change(const struct Expr *p, enum field_type type)
{
u8 op;
if (type == FIELD_TYPE_SCALAR)
- return 1;
+ return true;
while (p->op == TK_UPLUS || p->op == TK_UMINUS) {
p = p->pLeft;
}
@@ -2136,13 +2135,13 @@ sql_expr_needs_no_type_change(const struct Expr *p, enum field_type type)
case TK_STRING:
return type == FIELD_TYPE_STRING;
case TK_BLOB:
- return 1;
+ return true;
case TK_COLUMN:
/* p cannot be part of a CHECK constraint. */
assert(p->iTable >= 0);
return p->iColumn < 0 && sql_type_is_numeric(type);
default:
- return 0;
+ return false;
}
}
index d892631d8..f200752ce 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3734,7 +3734,7 @@ int sqlite3ExprCanBeNull(const Expr *);
* is harmless. A false positive, however, can result in the wrong
* answer.
*/
-int
+bool
sql_expr_needs_no_type_change(const struct Expr *expr, enum field_type type);
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2019-02-01 16:39 ` n.pettik
@ 2019-02-05 15:08 ` Vladislav Shpilevoy
2019-02-05 17:46 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-02-05 15:08 UTC (permalink / raw)
To: tarantool-patches, n.pettik
Thanks for the fixes! See 5 comments below.
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index 5676d0720..db8577d1c 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -252,74 +244,57 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
> -/*
> - * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
> - * idx_affinity is the affinity of an indexed column. Return true
> - * if the index with affinity idx_affinity may be used to implement
> - * the comparison in pExpr.
> +/**
> + * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
> + * @param field_type is the type of an indexed column.
> + * @retval Return true if the index with @field_type may be used to
> + * implement the comparison in expr.
> */
> -int
> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
> +bool
> +sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
> {
> - char aff = comparisonAffinity(pExpr);
> - switch (aff) {
> - case AFFINITY_BLOB:
> - return 1;
> - case AFFINITY_TEXT:
> - return idx_affinity == AFFINITY_TEXT;
> + enum field_type type = expr_cmp_mutual_type(expr);
> + switch (type) {
> + case FIELD_TYPE_SCALAR:
> + return true;
> + case FIELD_TYPE_STRING:
> + return field_type == FIELD_TYPE_STRING;
1. But type is never == FIELD_TYPE_STRING here. Because
expr_cmp_mutual_type() returns either SCALAR or NUMBER.
I added an assertion type != FIELD_TYPE_STRING and it did
not fail in tests.
> default:
> - return sqlite3IsNumericAffinity(idx_affinity);
> + return sql_type_is_numeric(field_type);
2. Also, I do not understand, why not to just use
field_type1_contains_type2(). sql_expr_cmp_type_is_compatible()
says, that it returns true, if a column of type field_type can
be used to "implement comparison" in an expression. So
this is the same as
field_type1_contains_type2(field_type, expr_cmp_mutual_type(expr));
If I correctly understood what does it mean - "implement comparison".
Then I tried to replace sql_expr_cmp_type_is_compatible() with the
call above and the tests failed. Then I swapped the arguments:
field_type1_contains_type2(expr_cmp_mutual_type(expr), field_type);
And the tests passed. Why?
> }
> }
>
> diff --git a/src/box/sql/select.c b/src/box/sql/select.c
> index ef4514374..9843eb4a5 100644
> --- a/src/box/sql/select.c
> +++ b/src/box/sql/select.c
> @@ -1728,38 +1727,31 @@ generateColumnNames(Parse * pParse, /* Parser context */
> p = pEList->a[i].pExpr;
> if (NEVER(p == 0))
> continue;
> - switch (p->affinity) {
> - case AFFINITY_INTEGER:
> + if (p->op == TK_VARIABLE)
> + var_pos[var_count++] = i;
3. Why did you move these two lines out of 'switch:' clause? As I
understand, for field types INTEGER, NUMBER, STRING, SCALAR op
never == TK_VARIABLE. So this check now executes mostly without
necessity, and should be done only for FIELD_TYPE_BOOLEAN. I
did this and the tests passed:
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 9843eb4a5..d8ab9a40d 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1727,8 +1727,6 @@ generateColumnNames(Parse * pParse, /* Parser context */
p = pEList->a[i].pExpr;
if (NEVER(p == 0))
continue;
- if (p->op == TK_VARIABLE)
- var_pos[var_count++] = i;
switch (p->type) {
case FIELD_TYPE_INTEGER:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "INTEGER",
@@ -1747,6 +1745,8 @@ generateColumnNames(Parse * pParse, /* Parser context */
SQLITE_TRANSIENT);
break;
case FIELD_TYPE_BOOLEAN:
+ if (p->op == TK_VARIABLE)
+ var_pos[var_count++] = i;
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BOOLEAN",
SQLITE_TRANSIENT);
break;
> + switch (p->type) {
> + case FIELD_TYPE_INTEGER:
> sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "INTEGER",
> SQLITE_TRANSIENT);
> break;
> - case AFFINITY_REAL:
> + case FIELD_TYPE_NUMBER:
> sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "NUMERIC",
> SQLITE_TRANSIENT);
> break;
> - case AFFINITY_TEXT:
> + case FIELD_TYPE_STRING:
> sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "TEXT",
> SQLITE_TRANSIENT);
> break;
> - case AFFINITY_BLOB:
> + case FIELD_TYPE_SCALAR:
> sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BLOB",
> SQLITE_TRANSIENT);
> break;
> - default: ;
> - char *type;
> - /*
> - * For variables we set BOOLEAN type since
> - * unassigned bindings will be replaced
> - * with NULL automatically, i.e. without
> - * explicit call of sql_bind_value().
> - */
> - if (p->op == TK_VARIABLE) {
> - var_pos[var_count++] = i;
> - type = "BOOLEAN";
> - } else {
> - type = "UNKNOWN";
> - }
> - sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, type,
> + case FIELD_TYPE_BOOLEAN:
> + sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BOOLEAN",
> + SQLITE_TRANSIENT);
> + break;
> + default:
> + sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "UNKNOWN",
> SQLITE_TRANSIENT);
> }
> if (pEList->a[i].zName) {
> diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
> index 79999509c..047a77b68 100644
> --- a/src/box/sql/sqliteInt.h
> +++ b/src/box/sql/sqliteInt.h
> @@ -2091,8 +2093,8 @@ typedef int ynVar;
> struct Expr {
> u8 op; /* Operation performed by this node */
> union {
> - /** The affinity of the column. */
> - enum affinity_type affinity;
> + /** The Type of the column. */
4. 'Type' -> 'type'.
> + enum field_type type;
> /** Conflict action for RAISE() function. */
> enum on_conflict_action on_conflict_action;
> };
> @@ -4267,26 +4281,30 @@ sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
> /**
> - * Return the affinity character for a single column of a table.
> - * @param def space definition.
> - * @param idx column index.
> - * @retval AFFINITY
> + * Return the type of the expression pExpr.
> + *
> + * If pExpr is a column, a reference to a column via an 'AS' alias,
> + * or a sub-select with a column as the return value, then the
> + * type of that column is returned. Otherwise, type ANY is returned,
> + * indicating that the expression can feature any type.
> + *
> + * The WHERE clause expressions in the following statements all
> + * have an type:
> + *
> + * CREATE TABLE t1(a);
> + * SELECT * FROM t1 WHERE a;
> + * SELECT a AS b FROM t1 WHERE b;
> + * SELECT * FROM t1 WHERE (select a from t1);
> */
> -char
> -sqlite3TableColumnAffinity(struct space_def *def, int idx);
> -
> -char sqlite3ExprAffinity(Expr * pExpr);
> +enum field_type
> +sql_expr_type(Expr *pExpr);
5. Expr -> struct Expr.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 6/8] sql: replace affinity with field type in struct Expr
2019-02-05 15:08 ` Vladislav Shpilevoy
@ 2019-02-05 17:46 ` n.pettik
0 siblings, 0 replies; 48+ messages in thread
From: n.pettik @ 2019-02-05 17:46 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index 5676d0720..db8577d1c 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -252,74 +244,57 @@ sql_expr_coll(Parse *parse, Expr *p, bool *is_explicit_coll, uint32_t *coll_id)
>> -/*
>> - * pExpr is a comparison expression, eg. '=', '<', IN(...) etc.
>> - * idx_affinity is the affinity of an indexed column. Return true
>> - * if the index with affinity idx_affinity may be used to implement
>> - * the comparison in pExpr.
>> +/**
>> + * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
>> + * @param field_type is the type of an indexed column.
>> + * @retval Return true if the index with @field_type may be used to
>> + * implement the comparison in expr.
>> */
>> -int
>> -sqlite3IndexAffinityOk(Expr * pExpr, char idx_affinity)
>> +bool
>> +sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
>> {
>> - char aff = comparisonAffinity(pExpr);
>> - switch (aff) {
>> - case AFFINITY_BLOB:
>> - return 1;
>> - case AFFINITY_TEXT:
>> - return idx_affinity == AFFINITY_TEXT;
>> + enum field_type type = expr_cmp_mutual_type(expr);
>> + switch (type) {
>> + case FIELD_TYPE_SCALAR:
>> + return true;
>> + case FIELD_TYPE_STRING:
>> + return field_type == FIELD_TYPE_STRING;
>
> 1. But type is never == FIELD_TYPE_STRING here. Because
> expr_cmp_mutual_type() returns either SCALAR or NUMBER.
> I added an assertion type != FIELD_TYPE_STRING and it did
> not fail in tests.
Well, that is actually true. See below, I removed this function at all.
>
>> default:
>> - return sqlite3IsNumericAffinity(idx_affinity);
>> + return sql_type_is_numeric(field_type);
>
> 2. Also, I do not understand, why not to just use
> field_type1_contains_type2(). sql_expr_cmp_type_is_compatible()
> says, that it returns true, if a column of type field_type can
> be used to "implement comparison" in an expression. So
> this is the same as
>
> field_type1_contains_type2(field_type, expr_cmp_mutual_type(expr));
>
> If I correctly understood what does it mean - "implement comparison".
> Then I tried to replace sql_expr_cmp_type_is_compatible() with the
> call above and the tests failed. Then I swapped the arguments:
>
> field_type1_contains_type2(expr_cmp_mutual_type(expr), field_type);
>
> And the tests passed. Why?
I guess because expr_cmp_mutal_type returns SCALAR or NUMBER
as you noticed. The first one is compatible with all scalar types and not
vice versa. So ..._type2(expr_cmp_mutual_type(expr), field_type) is not
the same as …_type2(field_type, expr_cmp_mutual_type(expr));
I’ve made comparative table and …_type2(expr_cmp_mutal_type, field_type)
gives us the same results as sql_expr_type_is_compatible.
Hence, I will remove this function and use …_type2() instead.
Diff:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index db8577d1c..36da1ecdb 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -252,12 +252,7 @@ sql_type_result(enum field_type lhs, enum field_type rhs)
return FIELD_TYPE_SCALAR;
}
-/**
- * pExpr is a comparison operator. Return the type affinity
- * that should be applied to both operands prior to doing
- * the comparison.
- */
-static enum field_type
+enum field_type
expr_cmp_mutual_type(struct Expr *pExpr)
{
assert(pExpr->op == TK_EQ || pExpr->op == TK_IN || pExpr->op == TK_LT ||
@@ -278,25 +273,6 @@ expr_cmp_mutual_type(struct Expr *pExpr)
return type;
}
-/**
- * @param expr is a comparison expression, eg. '=', '<', IN(...) etc.
- * @param field_type is the type of an indexed column.
- * @retval Return true if the index with @field_type may be used to
- * implement the comparison in expr.
- */
-bool
-sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type field_type)
-{
- enum field_type type = expr_cmp_mutual_type(expr);
- switch (type) {
- case FIELD_TYPE_SCALAR:
- return true;
- case FIELD_TYPE_STRING:
- return field_type == FIELD_TYPE_STRING;
- default:
- return sql_type_is_numeric(field_type);
- }
-}
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 047a77b68..61f316550 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -4284,8 +4284,13 @@ sql_emit_table_types(struct Vdbe *v, struct space_def *def, int reg);
enum field_type
sql_type_result(enum field_type lhs, enum field_type rhs);
-bool
-sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type);
+/**
+ * pExpr is a comparison operator. Return the type affinity
+ * that should be applied to both operands prior to doing
+ * the comparison.
+ */
+enum field_type
+expr_cmp_mutual_type(struct Expr *pExpr);
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 219009ef8..6a47f8c99 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -298,7 +298,9 @@ whereScanNext(WhereScan * pScan)
/* Verify the affinity and collating sequence match */
if ((pTerm->eOperator & WO_ISNULL) == 0) {
pX = pTerm->pExpr;
- if (!sql_expr_cmp_type_is_compatible(pX, pScan->idx_type))
+ enum field_type expr_type =
+ expr_cmp_mutual_type(pX);
+ if (!field_type1_contains_type2(expr_type, pScan->idx_type))
continue;
if (pScan->is_column_seen) {
Parse *pParse =
@@ -690,7 +692,6 @@ termCanDriveIndex(WhereTerm * pTerm, /* WHERE clause term to check */
Bitmask notReady /* Tables in outer loops of the join */
)
{
- char aff;
if (pTerm->leftCursor != pSrc->iCursor)
return 0;
if ((pTerm->eOperator & WO_EQ) == 0)
@@ -699,8 +700,9 @@ termCanDriveIndex(WhereTerm * pTerm, /* WHERE clause term to check */
return 0;
if (pTerm->u.leftColumn < 0)
return 0;
- aff = pSrc->pTab->aCol[pTerm->u.leftColumn].affinity;
- if (!sql_expr_cmp_type_is_compatible(pTerm->pExpr, aff))
+ enum field_type type = pSrc->pTab->def->fields[pTerm->u.leftColumn].type;
+ enum field_type expr_type = expr_cmp_mutual_type(pTerm->pExpr);
+ if (!field_type1_contains_type2(expr_type, type))
return 0;
return 1;
}
(The last change is related to dead code).
>> diff --git a/src/box/sql/select.c b/src/box/sql/select.c
>> index ef4514374..9843eb4a5 100644
>> --- a/src/box/sql/select.c
>> +++ b/src/box/sql/select.c
>> @@ -1728,38 +1727,31 @@ generateColumnNames(Parse * pParse, /* Parser context */
>> p = pEList->a[i].pExpr;
>> if (NEVER(p == 0))
>> continue;
>> - switch (p->affinity) {
>> - case AFFINITY_INTEGER:
>> + if (p->op == TK_VARIABLE)
>> + var_pos[var_count++] = i;
>
> 3. Why did you move these two lines out of 'switch:' clause? As I
> understand, for field types INTEGER, NUMBER, STRING, SCALAR op
> never == TK_VARIABLE. So this check now executes mostly without
> necessity, and should be done only for FIELD_TYPE_BOOLEAN. I
> did this and the tests passed:
You are absolutely right, just wanted to make it look better
(indeed, not the best try). Returned back:
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 9843eb4a5..d8ab9a40d 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1727,8 +1727,6 @@ generateColumnNames(Parse * pParse, /* Parser context */
p = pEList->a[i].pExpr;
if (NEVER(p == 0))
continue;
- if (p->op == TK_VARIABLE)
- var_pos[var_count++] = i;
switch (p->type) {
case FIELD_TYPE_INTEGER:
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "INTEGER",
@@ -1747,6 +1745,8 @@ generateColumnNames(Parse * pParse, /* Parser context */
SQLITE_TRANSIENT);
break;
case FIELD_TYPE_BOOLEAN:
+ if (p->op == TK_VARIABLE)
+ var_pos[var_count++] = i;
sqlite3VdbeSetColName(v, i, COLNAME_DECLTYPE, "BOOLEAN",
SQLITE_TRANSIENT);
break;
>> diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
>> index 79999509c..047a77b68 100644
>> --- a/src/box/sql/sqliteInt.h
>> +++ b/src/box/sql/sqliteInt.h
>> @@ -2091,8 +2093,8 @@ typedef int ynVar;
>> struct Expr {
>> u8 op; /* Operation performed by this node */
>> union {
>> - /** The affinity of the column. */
>> - enum affinity_type affinity;
>> + /** The Type of the column. */
>
> 4. 'Type' -> 'type’.
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 047a77b68..6e4ead573 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -2093,7 +2093,7 @@ typedef int ynVar;
struct Expr {
u8 op; /* Operation performed by this node */
union {
- /** The Type of the column. */
+ /** The type of the column. */
>> @@ -4267,26 +4281,30 @@ sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
>> /**
>> - * Return the affinity character for a single column of a table.
>> - * @param def space definition.
>> - * @param idx column index.
>> - * @retval AFFINITY
>> + * Return the type of the expression pExpr.
>> + *
>> + * If pExpr is a column, a reference to a column via an 'AS' alias,
>> + * or a sub-select with a column as the return value, then the
>> + * type of that column is returned. Otherwise, type ANY is returned,
>> + * indicating that the expression can feature any type.
>> + *
>> + * The WHERE clause expressions in the following statements all
>> + * have an type:
>> + *
>> + * CREATE TABLE t1(a);
>> + * SELECT * FROM t1 WHERE a;
>> + * SELECT a AS b FROM t1 WHERE b;
>> + * SELECT * FROM t1 WHERE (select a from t1);
>> */
>> -char
>> -sqlite3TableColumnAffinity(struct space_def *def, int idx);
>> -
>> -char sqlite3ExprAffinity(Expr * pExpr);
>> +enum field_type
>> +sql_expr_type(Expr *pExpr);
>
> 5. Expr -> struct Expr.
@@ -4304,7 +4304,7 @@ sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type);
* SELECT * FROM t1 WHERE (select a from t1);
*/
enum field_type
-sql_expr_type(Expr *pExpr);
+sql_expr_type(struct Expr *pExpr);
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] [PATCH 7/8] sql: clean-up affinity from SQL source code
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (5 preceding siblings ...)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 6/8] sql: replace affinity with field type in struct Expr Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
2018-12-28 9:34 ` [tarantool-patches] [PATCH 8/8] Remove affinity from field definition Nikita Pettik
` (2 subsequent siblings)
9 siblings, 1 reply; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Replace remains of affinity usage in SQL parser, query optimizer and
VDBE. Don't add affinity to field definition when table is encoded into
msgpack. Remove field type <-> affinity converters, since now we can
operate directly on field type.
Part of #3698
---
src/box/sql.c | 6 +-----
src/box/sql/build.c | 52 ------------------------------------------------
src/box/sql/expr.c | 11 +++++-----
src/box/sql/insert.c | 39 ++++++------------------------------
src/box/sql/select.c | 17 +++++-----------
src/box/sql/sqliteInt.h | 53 -------------------------------------------------
src/box/sql/vdbe.c | 4 ++--
src/box/sql/where.c | 10 ----------
src/box/sql/wherecode.c | 29 ++++++++++-----------------
src/box/sql/whereexpr.c | 3 ++-
test/sql/types.result | 15 +++++++-------
test/sql/upgrade.result | 6 +++---
12 files changed, 42 insertions(+), 203 deletions(-)
diff --git a/src/box/sql.c b/src/box/sql.c
index a498cd8fe..8e2f5b6c7 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -997,7 +997,7 @@ sql_encode_table(struct region *region, struct Table *table, uint32_t *size)
uint32_t cid = def->fields[i].coll_id;
struct field_def *field = &def->fields[i];
const char *default_str = field->default_value;
- int base_len = 5;
+ int base_len = 4;
if (cid != COLL_NONE)
base_len += 1;
if (default_str != NULL)
@@ -1009,10 +1009,6 @@ sql_encode_table(struct region *region, struct Table *table, uint32_t *size)
assert(def->fields[i].is_nullable ==
action_is_nullable(def->fields[i].nullable_action));
mpstream_encode_str(&stream, field_type_strs[field->type]);
- mpstream_encode_str(&stream, "affinity");
- enum affinity_type aff =
- sql_field_type_to_affinity(def->fields[i].type);
- mpstream_encode_uint(&stream, aff);
mpstream_encode_str(&stream, "is_nullable");
mpstream_encode_bool(&stream, def->fields[i].is_nullable);
mpstream_encode_str(&stream, "nullable_action");
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index 2885bb6d5..9eaa4300a 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -485,58 +485,6 @@ sql_field_retrieve(Parse *parser, Table *table, uint32_t id)
return field;
}
-enum field_type
-sql_affinity_to_field_type(enum affinity_type affinity)
-{
- switch (affinity) {
- case AFFINITY_INTEGER:
- return FIELD_TYPE_INTEGER;
- case AFFINITY_REAL:
- return FIELD_TYPE_NUMBER;
- case AFFINITY_TEXT:
- return FIELD_TYPE_STRING;
- case AFFINITY_BLOB:
- return FIELD_TYPE_SCALAR;
- case AFFINITY_UNDEFINED:
- return FIELD_TYPE_ANY;
- default:
- unreachable();
- }
-}
-
-enum affinity_type
-sql_field_type_to_affinity(enum field_type field_type)
-{
- switch (field_type) {
- case FIELD_TYPE_INTEGER:
- case FIELD_TYPE_UNSIGNED:
- return AFFINITY_INTEGER;
- case FIELD_TYPE_NUMBER:
- return AFFINITY_REAL;
- case FIELD_TYPE_STRING:
- return AFFINITY_TEXT;
- case FIELD_TYPE_SCALAR:
- return AFFINITY_BLOB;
- case FIELD_TYPE_ANY:
- return AFFINITY_UNDEFINED;
- default:
- unreachable();
- }
-}
-
-char *
-sql_affinity_str_to_field_type_str(const char *affinity_str)
-{
- if (affinity_str == NULL)
- return NULL;
- size_t len = strlen(affinity_str) + 1;
- char *type_str = (char *) sqlite3DbMallocRaw(sql_get(), len);
- for (uint32_t i = 0; i < len - 1; ++i)
- type_str[i] = sql_affinity_to_field_type(affinity_str[i]);
- type_str[len - 1] = '\0';
- return type_str;
-}
-
/*
* Add a new column to the table currently being constructed.
*
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 32606dac3..22b64b526 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -311,8 +311,7 @@ binaryCompareP5(Expr * pExpr1, Expr * pExpr2, int jumpIfNull)
{
enum field_type lhs = sql_expr_type(pExpr2);
enum field_type rhs = sql_expr_type(pExpr1);
- u8 type_mask = sql_field_type_to_affinity(sql_type_result(rhs, lhs)) |
- (u8) jumpIfNull;
+ u8 type_mask = sql_type_result(rhs, lhs) | (u8) jumpIfNull;
return type_mask;
}
@@ -2606,9 +2605,9 @@ exprINAffinity(Parse * pParse, Expr * pExpr)
if (pSelect) {
struct Expr *e = pSelect->pEList->a[i].pExpr;
enum field_type rhs = sql_expr_type(e);
- zRet[i] = sql_field_type_to_affinity(sql_type_result(rhs, lhs));
+ zRet[i] = sql_type_result(rhs, lhs);
} else {
- zRet[i] = sql_field_type_to_affinity(lhs);
+ zRet[i] = lhs;
}
}
zRet[nVal] = '\0';
@@ -2826,6 +2825,7 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
jmpIfDynamic = -1;
}
r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
+ assert(lhs_type < field_type_MAX);
sqlite3VdbeAddOp4(v, OP_MakeRecord, r3,
1, r2,
(char *) &lhs_type,
@@ -3142,8 +3142,7 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- char *type_str = sql_affinity_str_to_field_type_str(zAff);
- sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, type_str, nVector);
+ sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, zAff, nVector);
if (destIfFalse == destIfNull) {
/* Combine Step 3 and Step 5 into a single opcode */
sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index acc7712f6..cc06638cb 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -41,33 +41,6 @@
#include "bit/bit.h"
#include "box/box.h"
-char *
-sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
- struct index_def *idx_def)
-{
- uint32_t column_count = idx_def->key_def->part_count;
- char *aff = (char *)sqlite3DbMallocRaw(db, column_count + 1);
- if (aff == NULL)
- return NULL;
- /*
- * Table may occasionally come from non-SQL API, so lets
- * gentle process this case by setting default affinity
- * for it.
- */
- if (space_def->fields == NULL) {
- memset(aff, AFFINITY_BLOB, column_count);
- } else {
- for (uint32_t i = 0; i < column_count; i++) {
- aff[i] = sql_space_index_part_affinity(space_def,
- idx_def, i);
- if (aff[i] == AFFINITY_UNDEFINED)
- aff[i] = AFFINITY_BLOB;
- }
- }
- aff[column_count] = '\0';
- return aff;
-}
-
char *
sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def)
{
@@ -1247,12 +1220,12 @@ xferOptimization(Parse * pParse, /* Parser context */
if (dest->def->field_count != src->def->field_count)
return 0;
for (i = 0; i < (int)dest->def->field_count; i++) {
- enum affinity_type dest_affinity =
- dest->def->fields[i].affinity;
- enum affinity_type src_affinity =
- src->def->fields[i].affinity;
- /* Affinity must be the same on all columns. */
- if (dest_affinity != src_affinity)
+ enum field_type dest_type =
+ dest->def->fields[i].type;
+ enum field_type src_type =
+ src->def->fields[i].type;
+ /* Type must be the same on all columns. */
+ if (dest_type != src_type)
return 0;
uint32_t id;
if (sql_column_collation(dest->def, i, &id) !=
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index cc3e2f2fd..f3008094b 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1202,12 +1202,9 @@ selectInnerLoop(Parse * pParse, /* The parser context */
int r1 = sqlite3GetTempReg(pParse);
assert(sqlite3Strlen30(pDest->zAffSdst) ==
(unsigned int)nResultCol);
- char *type_str =
- sql_affinity_str_to_field_type_str(
- pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
- nResultCol, r1, type_str,
- P4_DYNAMIC);
+ nResultCol, r1,
+ pDest->zAffSdst, nResultCol);
sqlite3ExprCacheAffinityChange(pParse,
regResult,
nResultCol);
@@ -1629,10 +1626,8 @@ generateSortTail(Parse * pParse, /* Parsing context */
assert((unsigned int)nColumn ==
sqlite3Strlen30(pDest->zAffSdst));
- const char *type_str =
- sql_affinity_str_to_field_type_str(pDest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
- regTupleid, type_str, P4_DYNAMIC);
+ regTupleid, pDest->zAffSdst, nColumn);
sqlite3ExprCacheAffinityChange(pParse, regRow, nColumn);
sqlite3VdbeAddOp2(v, OP_IdxInsert, regTupleid, pDest->reg_eph);
break;
@@ -1974,7 +1969,6 @@ sqlite3SelectAddColumnTypeAndCollation(Parse * pParse, /* Parsing contexts */
enum field_type type = sql_expr_type(p);
if (type == FIELD_TYPE_ANY)
type = FIELD_TYPE_SCALAR;
- pTab->def->fields[i].affinity = sql_field_type_to_affinity(type);
pTab->def->fields[i].type = type;
bool is_found;
uint32_t coll_id;
@@ -3067,10 +3061,9 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
int r1;
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
- const char *type_str =
- sql_affinity_str_to_field_type_str(dest->zAffSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
- in->nSdst, r1, type_str, P4_DYNAMIC);
+ in->nSdst, r1, dest->zAffSdst,
+ in->nSdst);
sqlite3ExprCacheAffinityChange(parse, in->iSdst,
in->nSdst);
sqlite3VdbeAddOp2(v, OP_IdxInsert, r1, dest->reg_eph);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 7d1159345..807ca16c6 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -3431,19 +3431,6 @@ sql_create_view(struct Parse *parse_context, struct Token *begin,
struct Token *name, struct ExprList *aliases,
struct Select *select, bool if_exists);
-/**
- * Helper to convert SQLite affinity to corresponding
- * Tarantool field type.
- **/
-enum field_type
-sql_affinity_to_field_type(enum affinity_type affinity);
-
-enum affinity_type
-sql_field_type_to_affinity(enum field_type field_type);
-
-char *
-sql_affinity_str_to_field_type_str(const char *affinity_str);
-
/**
* Compile view, i.e. create struct Select from
* 'CREATE VIEW...' string, and assign cursors to each table from
@@ -4215,33 +4202,6 @@ int sqlite3VarintLen(u64 v);
#define getVarint sqlite3GetVarint
#define putVarint sqlite3PutVarint
-/**
- * Return a pointer to the column affinity string associated with
- * given index. A column affinity string has one character for
- * each column in the table, according to the affinity of the
- * column:
- *
- * Character Column affinity
- * ------------------------------
- * 'A' BLOB
- * 'B' TEXT
- * 'C' NUMERIC
- * 'D' INTEGER
- * 'F' REAL
- *
- * Memory for the buffer containing the column index affinity string
- * is allocated on heap.
- *
- * @param db Database handle.
- * @param space_def Definition of space index belongs to.
- * @param idx_def Definition of index from which affinity
- * to be extracted.
- * @retval Allocated affinity string, or NULL on OOM.
- */
-char *
-sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
- struct index_def *idx_def);
-
/** Return string consisting of fields types of given index. */
char *
sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
@@ -4631,19 +4591,6 @@ int
sql_stat4_column(struct sqlite3 *db, const char *record, uint32_t col_num,
sqlite3_value **res);
-/**
- * Return the affinity for a single column of an index.
- *
- * @param def Definition of space @idx belongs to.
- * @param idx Index to be investigated.
- * @param partno Affinity of this part to be returned.
- *
- * @retval Affinity of @partno index part.
- */
-enum affinity_type
-sql_space_index_part_affinity(struct space_def *def, struct index_def *idx,
- uint32_t partno);
-
/*
* The interface to the LEMON-generated parser
*/
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 369fb4b79..24cc08e8f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2165,7 +2165,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
} else {
/* Neither operand is NULL. Do a comparison. */
affinity = pOp->p5 & AFFINITY_MASK;
- if (affinity>=AFFINITY_INTEGER) {
+ if (sql_type_is_numeric(affinity)) {
if ((flags1 | flags3)&MEM_Str) {
if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
applyNumericAffinity(pIn1,0);
@@ -2193,7 +2193,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- } else if (affinity==AFFINITY_TEXT) {
+ } else if (affinity == FIELD_TYPE_STRING) {
if ((flags1 & MEM_Str)==0 && (flags1 & (MEM_Int|MEM_Real))!=0) {
testcase( pIn1->flags & MEM_Int);
testcase( pIn1->flags & MEM_Real);
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index ac5390f92..88c45aaa3 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -701,7 +701,6 @@ termCanDriveIndex(WhereTerm * pTerm, /* WHERE clause term to check */
return 0;
if (pTerm->u.leftColumn < 0)
return 0;
- aff = pSrc->pTab->aCol[pTerm->u.leftColumn].affinity;
if (!sqlite3IndexAffinityOk(pTerm->pExpr, aff))
return 0;
return 1;
@@ -1139,15 +1138,6 @@ whereRangeAdjust(WhereTerm * pTerm, LogEst nNew)
return nRet;
}
-enum affinity_type
-sql_space_index_part_affinity(struct space_def *def, struct index_def *idx,
- uint32_t partno)
-{
- assert(partno < idx->key_def->part_count);
- uint32_t fieldno = idx->key_def->parts[partno].fieldno;
- return def->fields[fieldno].affinity;
-}
-
/*
* This function is called to estimate the number of rows visited by a
* range-scan on a skip-scan index. For example:
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index 06335bcf7..759214b22 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -387,19 +387,18 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
/* Adjust base and n to skip over AFFINITY_BLOB entries at the beginning
* and end of the affinity string.
*/
- while (n > 0 && zAff[0] == AFFINITY_BLOB) {
+ while (n > 0 && zAff[0] == FIELD_TYPE_SCALAR) {
n--;
base++;
zAff++;
}
- while (n > 1 && zAff[n - 1] == AFFINITY_BLOB) {
+ while (n > 1 && zAff[n - 1] == FIELD_TYPE_SCALAR) {
n--;
}
if (n > 0) {
- const char *type_str = sql_affinity_str_to_field_type_str(zAff);
- sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, type_str, n);
+ sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, zAff, n);
sqlite3ExprCacheAffinityChange(pParse, base, n);
}
}
@@ -421,12 +420,11 @@ updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
{
int i;
for (i = 0; i < n; i++) {
- enum field_type type = sql_affinity_to_field_type(zAff[i]);
Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
enum field_type expr_type = sql_expr_type(p);
- if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
- sql_expr_needs_type_change(p, type)) {
- zAff[i] = AFFINITY_BLOB;
+ if (sql_type_result(expr_type, zAff[i]) == FIELD_TYPE_SCALAR ||
+ sql_expr_needs_type_change(p, zAff[i])) {
+ zAff[i] = FIELD_TYPE_SCALAR;
}
}
}
@@ -709,11 +707,7 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
nReg = pLoop->nEq + nExtraReg;
pParse->nMem += nReg;
-
- struct space *space = space_by_id(idx_def->space_id);
- assert(space != NULL);
- char *zAff = sql_space_index_affinity_str(pParse->db, space->def,
- idx_def);
+ char *zAff = sql_index_type_str(pParse->db, idx_def);
assert(zAff != 0 || pParse->db->mallocFailed);
if (nSkip) {
@@ -767,7 +761,7 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
* affinity of the comparison has been applied to the value.
*/
if (zAff)
- zAff[j] = AFFINITY_BLOB;
+ zAff[j] = FIELD_TYPE_SCALAR;
}
} else if ((pTerm->eOperator & WO_ISNULL) == 0) {
Expr *pRight = pTerm->pExpr->pRight;
@@ -779,14 +773,13 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
if (zAff) {
enum field_type type =
sql_expr_type(pRight);
- enum field_type idx_type =
- sql_affinity_to_field_type(zAff[j]);
+ enum field_type idx_type = zAff[j];
if (sql_type_result(type, idx_type) ==
FIELD_TYPE_SCALAR) {
- zAff[j] = AFFINITY_BLOB;
+ zAff[j] = FIELD_TYPE_SCALAR;
}
if (sql_expr_needs_type_change(pRight, idx_type))
- zAff[j] = AFFINITY_BLOB;
+ zAff[j] = FIELD_TYPE_SCALAR;
}
}
}
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 6ffd5ae84..96838f4b2 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -284,7 +284,8 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
Vdbe *pReprepare = pParse->pReprepare;
int iCol = pRight->iColumn;
pVal =
- sqlite3VdbeGetBoundValue(pReprepare, iCol, AFFINITY_BLOB);
+ sqlite3VdbeGetBoundValue(pReprepare, iCol,
+ FIELD_TYPE_SCALAR);
if (pVal && sqlite3_value_type(pVal) == SQLITE_TEXT) {
z = (char *)sqlite3_value_text(pVal);
}
diff --git a/test/sql/types.result b/test/sql/types.result
index 915a6341a..df4dc151e 100644
--- a/test/sql/types.result
+++ b/test/sql/types.result
@@ -33,20 +33,19 @@ box.sql.execute("CREATE TABLE t1 (id TEXT PRIMARY KEY, a REAL, b INT, c TEXT, d
...
box.space.T1:format()
---
-- [{'affinity': 66, 'type': 'string', 'nullable_action': 'abort', 'name': 'ID', 'is_nullable': false},
- {'affinity': 69, 'type': 'number', 'nullable_action': 'none', 'name': 'A', 'is_nullable': true},
- {'affinity': 68, 'type': 'integer', 'nullable_action': 'none', 'name': 'B', 'is_nullable': true},
- {'affinity': 66, 'type': 'string', 'nullable_action': 'none', 'name': 'C', 'is_nullable': true},
- {'affinity': 65, 'type': 'scalar', 'nullable_action': 'none', 'name': 'D', 'is_nullable': true}]
+- [{'type': 'string', 'nullable_action': 'abort', 'name': 'ID', 'is_nullable': false},
+ {'type': 'number', 'nullable_action': 'none', 'name': 'A', 'is_nullable': true},
+ {'type': 'integer', 'nullable_action': 'none', 'name': 'B', 'is_nullable': true},
+ {'type': 'string', 'nullable_action': 'none', 'name': 'C', 'is_nullable': true},
+ {'type': 'scalar', 'nullable_action': 'none', 'name': 'D', 'is_nullable': true}]
...
box.sql.execute("CREATE VIEW v1 AS SELECT b + a, b - a FROM t1;")
---
...
box.space.V1:format()
---
-- [{'affinity': 69, 'type': 'number', 'nullable_action': 'none', 'name': 'b + a',
- 'is_nullable': true}, {'affinity': 69, 'type': 'number', 'nullable_action': 'none',
- 'name': 'b - a', 'is_nullable': true}]
+- [{'type': 'number', 'nullable_action': 'none', 'name': 'b + a', 'is_nullable': true},
+ {'type': 'number', 'nullable_action': 'none', 'name': 'b - a', 'is_nullable': true}]
...
-- gh-2494: index's part also features correct declared type.
--
diff --git a/test/sql/upgrade.result b/test/sql/upgrade.result
index 79c7eb245..02ab9b42b 100644
--- a/test/sql/upgrade.result
+++ b/test/sql/upgrade.result
@@ -80,12 +80,12 @@ box.sql.execute("CREATE TRIGGER t2t AFTER INSERT ON t BEGIN INSERT INTO t_out VA
...
box.space._space.index['name']:get('T')
---
-- [513, 1, 'T', 'memtx', 1, {}, [{'affinity': 68, 'type': 'integer', 'nullable_action': 'abort',
- 'name': 'X', 'is_nullable': false}]]
+- [513, 1, 'T', 'memtx', 1, {}, [{'type': 'integer', 'nullable_action': 'abort', 'name': 'X',
+ 'is_nullable': false}]]
...
box.space._space.index['name']:get('T_OUT')
---
-- [514, 1, 'T_OUT', 'memtx', 1, {}, [{'affinity': 68, 'type': 'integer', 'nullable_action': 'abort',
+- [514, 1, 'T_OUT', 'memtx', 1, {}, [{'type': 'integer', 'nullable_action': 'abort',
'name': 'X', 'is_nullable': false}]]
...
t1t = box.space._trigger:get('T1T')
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2018-12-28 9:34 ` [tarantool-patches] [PATCH 7/8] sql: clean-up affinity from SQL source code Nikita Pettik
@ 2018-12-29 17:42 ` Vladislav Shpilevoy
2019-01-16 14:26 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2018-12-29 17:42 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thanks for the patch! See 5 comments below.
On 28/12/2018 12:34, Nikita Pettik wrote:
> Replace remains of affinity usage in SQL parser, query optimizer and
> VDBE. Don't add affinity to field definition when table is encoded into
> msgpack. Remove field type <-> affinity converters, since now we can
> operate directly on field type.
>
> Part of #3698
> ---
> src/box/sql.c | 6 +-----
> src/box/sql/build.c | 52 ------------------------------------------------
> src/box/sql/expr.c | 11 +++++-----
> src/box/sql/insert.c | 39 ++++++------------------------------
> src/box/sql/select.c | 17 +++++-----------
> src/box/sql/sqliteInt.h | 53 -------------------------------------------------
> src/box/sql/vdbe.c | 4 ++--
> src/box/sql/where.c | 10 ----------
> src/box/sql/wherecode.c | 29 ++++++++++-----------------
> src/box/sql/whereexpr.c | 3 ++-
> test/sql/types.result | 15 +++++++-------
> test/sql/upgrade.result | 6 +++---
> 12 files changed, 42 insertions(+), 203 deletions(-)
>
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index 32606dac3..22b64b526 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -311,8 +311,7 @@ binaryCompareP5(Expr * pExpr1, Expr * pExpr2, int jumpIfNull)
> {
> enum field_type lhs = sql_expr_type(pExpr2);
> enum field_type rhs = sql_expr_type(pExpr1);
> - u8 type_mask = sql_field_type_to_affinity(sql_type_result(rhs, lhs)) |
> - (u8) jumpIfNull;
> + u8 type_mask = sql_type_result(rhs, lhs) | (u8) jumpIfNull;
1. Are you sure that we can | jumpIfNull with enum field_type? Look at
this code:
#define SQLITE_KEEPNULL 0x08 /* Used by vector == or <> */
#define SQLITE_JUMPIFNULL 0x10 /* jumps if either operand is NULL */
#define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
#define SQLITE_NULLEQ 0x80 /* NULL=NULL */
#define SQLITE_NOTNULL 0x90 /* Assert that operands are never NULL */
SQLite states that these values can be safely ORed with type, but
SQLITE_KEEPNULL == FIELD_TYPE_MAP - mess.
> return type_mask;
> }
> diff --git a/src/box/sql/select.c b/src/box/sql/select.c
> index cc3e2f2fd..f3008094b 100644
> --- a/src/box/sql/select.c
> +++ b/src/box/sql/select.c
> @@ -3067,10 +3061,9 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
> int r1;
> testcase(in->nSdst > 1);
> r1 = sqlite3GetTempReg(parse);
> - const char *type_str =
> - sql_affinity_str_to_field_type_str(dest->zAffSdst);
> sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
> - in->nSdst, r1, type_str, P4_DYNAMIC);
> + in->nSdst, r1, dest->zAffSdst,
> + in->nSdst);
2. As we learned from the previous patch, p4 here is not
a length. It is a type of memory - static or dynamic.
> sqlite3ExprCacheAffinityChange(parse, in->iSdst,
> in->nSdst);
> sqlite3VdbeAddOp2(v, OP_IdxInsert, r1, dest->reg_eph);
> diff --git a/src/box/sql/where.c b/src/box/sql/where.c
> index ac5390f92..88c45aaa3 100644
> --- a/src/box/sql/where.c
> +++ b/src/box/sql/where.c
> @@ -701,7 +701,6 @@ termCanDriveIndex(WhereTerm * pTerm, /* WHERE clause term to check */
> return 0;
> if (pTerm->u.leftColumn < 0)
> return 0;
> - aff = pSrc->pTab->aCol[pTerm->u.leftColumn].affinity;
> if (!sqlite3IndexAffinityOk(pTerm->pExpr, aff))
3. This function does not exist since the previous commit. Looks
like this code is not even compiled.
> return 0;
> return 1;
4. AFFINITY_MASK still exists.
5. wherecode.c and vdbeaux.c still use AFFINITY_BLOB in comments.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2018-12-29 17:42 ` [tarantool-patches] " Vladislav Shpilevoy
@ 2019-01-16 14:26 ` n.pettik
2019-01-22 15:41 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-16 14:26 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index 32606dac3..22b64b526 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -311,8 +311,7 @@ binaryCompareP5(Expr * pExpr1, Expr * pExpr2, int jumpIfNull)
>> {
>> enum field_type lhs = sql_expr_type(pExpr2);
>> enum field_type rhs = sql_expr_type(pExpr1);
>> - u8 type_mask = sql_field_type_to_affinity(sql_type_result(rhs, lhs)) |
>> - (u8) jumpIfNull;
>> + u8 type_mask = sql_type_result(rhs, lhs) | (u8) jumpIfNull;
>
> 1. Are you sure that we can | jumpIfNull with enum field_type? Look at
> this code:
>
> #define SQLITE_KEEPNULL 0x08 /* Used by vector == or <> */
> #define SQLITE_JUMPIFNULL 0x10 /* jumps if either operand is NULL */
> #define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
> #define SQLITE_NULLEQ 0x80 /* NULL=NULL */
> #define SQLITE_NOTNULL 0x90 /* Assert that operands are never NULL */
>
> SQLite states that these values can be safely ORed with type, but
> SQLITE_KEEPNULL == FIELD_TYPE_MAP - mess.
Thx, it is really mess. Funny thing - it didn’t break anything :)
Fixed:
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index d4425eaa0..822cc40df 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1807,7 +1801,7 @@ struct Savepoint {
* operator is NULL. It is added to certain comparison operators to
* prove that the operands are always NOT NULL.
*/
-#define SQLITE_KEEPNULL 0x08 /* Used by vector == or <> */
+#define SQLITE_KEEPNULL 0x40 /* Used by vector == or <> */
>
>> return type_mask;
>> }
>> diff --git a/src/box/sql/select.c b/src/box/sql/select.c
>> index cc3e2f2fd..f3008094b 100644
>> --- a/src/box/sql/select.c
>> +++ b/src/box/sql/select.c
>> @@ -3067,10 +3061,9 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
>> int r1;
>> testcase(in->nSdst > 1);
>> r1 = sqlite3GetTempReg(parse);
>> - const char *type_str =
>> - sql_affinity_str_to_field_type_str(dest->zAffSdst);
>> sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
>> - in->nSdst, r1, type_str, P4_DYNAMIC);
>> + in->nSdst, r1, dest->zAffSdst,
>> + in->nSdst);
>
> 2. As we learned from the previous patch, p4 here is not
> a length. It is a type of memory - static or dynamic.
In fact, it’s length and type of allocation :)
Anyway, I’ve reworked this place (alongside with other similar to this)
since now we are using array of field types which are not terminated
with \0.
Diff:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index 07a7349a2..db7ddad06 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -106,6 +106,19 @@ sql_expr_type(struct Expr *pExpr)
return pExpr->type;
}
+enum field_type *
+field_type_sequence_dup(struct Parse *parse, enum field_type *types,
+ uint32_t len)
+{
+ uint32_t sz = (len + 1) * sizeof(enum field_type);
+ enum field_type *ret_types = sqlite3DbMallocRaw(parse->db, sz);
+ if (ret_types == NULL)
+ return NULL;
+ memcpy(ret_types, types, sz);
+ ret_types[len] = field_type_MAX;
+ return ret_types;
+}
+
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index f3008094b..92b563e22 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -3061,9 +3064,12 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
int r1;
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
+ enum field_type *types =
+ field_type_sequence_dup(parse, dest->zAffSdst,
+ in->nSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
- in->nSdst, r1, dest->zAffSdst,
- in->nSdst);
+ in->nSdst, r1, (char *)types,
+ P4_DYNAMIC);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 807ca16c6..d4425eaa0 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -4242,6 +4253,13 @@ sql_index_type_is_ok(struct Expr *expr, enum field_type idx_type);
enum field_type
sql_expr_type(Expr *pExpr);
+/**
+ * This function duplicates first @len entries of types array
+ * and terminates new array with field_type_MAX member.
+ */
+enum field_type *
+field_type_sequence_dup(struct Parse *parse, enum field_type *types,
+ uint32_t len);
>
>> sqlite3ExprCacheAffinityChange(parse, in->iSdst,
>> in->nSdst);
>> sqlite3VdbeAddOp2(v, OP_IdxInsert, r1, dest->reg_eph);
>> diff --git a/src/box/sql/where.c b/src/box/sql/where.c
>> index ac5390f92..88c45aaa3 100644
>> --- a/src/box/sql/where.c
>> +++ b/src/box/sql/where.c
>> @@ -701,7 +701,6 @@ termCanDriveIndex(WhereTerm * pTerm, /* WHERE clause term to check */
>> return 0;
>> if (pTerm->u.leftColumn < 0)
>> return 0;
>> - aff = pSrc->pTab->aCol[pTerm->u.leftColumn].affinity;
>> if (!sqlite3IndexAffinityOk(pTerm->pExpr, aff))
>
> 3. This function does not exist since the previous commit. Looks
> like this code is not even compiled.
Yep, it is dead code under #ifndef SQLITE_OMIT_AUTOMATIC_INDEX
Auto-indexes haven’t been working since January of 2018.
I used IDE auto-refactoring tool and for some reason it missed this place.
Fixed in previous patch.
>
>> return 0;
>> return 1;
>
> 4. AFFINITY_MASK still exists.
Removed:
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index ef5a03087..c3f596d4f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2022,14 +2022,6 @@ case OP_Cast: { /* in1 */
* jump to address P2. Or if the SQLITE_STOREP2 flag is set in P5, then
* store the result of comparison in register P2.
*
- * The AFFINITY_MASK portion of P5 must be an affinity character -
- * AFFINITY_TEXT, AFFINITY_INTEGER, and so forth. An attempt is made
- * to coerce both inputs according to this affinity before the
- * comparison is made. If the AFFINITY_MASK is 0x00, then numeric
- * affinity is used. Note that the affinity conversions are stored
- * back into the input registers P1 and P3. So this opcode can cause
- * persistent changes to registers P1 and P3.
- *
* Once any conversions have taken place, and neither value is NULL,
* the values are compared. If both values are blobs then memcmp() is
* used to determine the results of the comparison. If both values
@@ -2045,6 +2037,10 @@ case OP_Cast: { /* in1 */
* of comparison is true. If either operand is NULL then the result is false.
* If neither operand is NULL the result is the same as it would be if
* the SQLITE_NULLEQ flag were omitted from P5.
+ * P5 also can contain type to be applied to operands. Note that
+ * the type conversions are stored back into the input registers
+ * P1 and P3. So this opcode can cause persistent changes to
+ * registers P1 and P3.
*
* If both SQLITE_STOREP2 and SQLITE_KEEPNULL flags are set then the
* content of r[P2] is only changed if the new value is NULL or 0 (false).
@@ -2072,14 +2068,6 @@ case OP_Cast: { /* in1 */
* reg(P3) is NULL then the take the jump. If the SQLITE_JUMPIFNULL
* bit is clear then fall through if either operand is NULL.
*
- * The AFFINITY_MASK portion of P5 must be an affinity character -
- * AFFINITY_TEXT, AFFINITY_INTEGER, and so forth. An attempt is made
- * to coerce both inputs according to this affinity before the
- * comparison is made. If the AFFINITY_MASK is 0x00, then numeric
- * affinity is used. Note that the affinity conversions are stored
- * back into the input registers P1 and P3. So this opcode can cause
- * persistent changes to registers P1 and P3.
- *
* Once any conversions have taken place, and neither value is NULL,
* the values are compared. If both values are blobs then memcmp() is
* used to determine the results of the comparison. If both values
@@ -2163,8 +2151,15 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
break;
}
} else {
- /* Neither operand is NULL. Do a comparison. */
- affinity = pOp->p5 & AFFINITY_MASK;
+ /*
+ * Neither operand is NULL. Do a comparison.
+ * 15 is 1111 in a binary form.
+ * Since all existing types can be fitted in 4 bits
+ * (field_type_MAX == 10), it is enough to 'and'
+ * p5 with this constant. Node that all other flags
+ * that can be stored in p5 are >= 16.
+ */
+ affinity = pOp->p5 & 15;
if (sql_type_is_numeric(affinity)) {
if ((flags1 | flags3)&MEM_Str) {
if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
>
> 5. wherecode.c and vdbeaux.c still use AFFINITY_BLOB in comments.
Cleaned:
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 08eaddbfc..dbde4d832 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -3492,8 +3492,7 @@ sqlite3VdbeDb(Vdbe * v)
/*
* Return a pointer to an sqlite3_value structure containing the value bound
* parameter iVar of VM v. Except, if the value is an SQL NULL, return
- * 0 instead. Unless it is NULL, apply affinity aff (one of the AFFINITY_*
- * constants) to the value before returning it.
+ * 0 instead. Unless it is NULL, apply type to the value before returning it.
*
* The returned value must be freed by the caller using sqlite3ValueFree().
*/
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index c28ff33f9..01c1a2c6c 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -367,9 +367,9 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
* Code an OP_ApplyType opcode to apply the column type string types
* to the n registers starting at base.
*
- * As an optimization, AFFINITY_BLOB entries (which are no-ops) at the
+ * As an optimization, SCALAR entries (which are no-ops) at the
* beginning and end of zAff are ignored. If all entries in zAff are
- * AFFINITY_BLOB, then no code gets generated.
+ * SCALAR, then no code gets generated.
*
* This routine makes its own copy of zAff so that the caller is free
* to modify zAff after this routine returns.
@@ -384,8 +384,9 @@ codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
}
assert(v != 0);
- /* Adjust base and n to skip over AFFINITY_BLOB entries at the beginning
- * and end of the affinity string.
+ /*
+ * Adjust base and n to skip over SCALAR entries at the
+ * beginning and end of the type sequence.
*/
while (n > 0 && zAff[0] == FIELD_TYPE_SCALAR) {
n--;
@@ -409,8 +410,8 @@ codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
* Expression pRight, which is the RHS of a comparison operation, is
* either a vector of n elements or, if n==1, a scalar expression.
* Before the comparison operation, affinity zAff is to be applied
- * to the pRight values. This function modifies characters within the
- * affinity string to AFFINITY_BLOB if either:
+ * to the pRight values. This function modifies entries within the
+ * field sequence to SCALAR if either:
*
* * the comparison will be performed with no affinity, or
* * the affinity change in zAff is guaranteed not to change the value.
@@ -669,7 +670,7 @@ codeEqualityTerm(Parse * pParse, /* The parsing context */
* copy of the column affinity string of the index allocated using
* sqlite3DbMalloc(). Except, entries in the copy of the string associated
* with equality constraints that use BLOB or NONE affinity are set to
- * AFFINITY_BLOB. This is to deal with SQL such as the following:
+ * SCALAR. This is to deal with SQL such as the following:
*
* CREATE TABLE t1(a TEXT PRIMARY KEY, b);
* SELECT ... FROM t1 AS t2, t1 WHERE t1.a = t2.b;
@@ -678,7 +679,7 @@ codeEqualityTerm(Parse * pParse, /* The parsing context */
* the right hand side of the equality constraint (t2.b) has BLOB/NONE affinity,
* no conversion should be attempted before using a t2.b value as part of
* a key to search the index. Hence the first byte in the returned affinity
- * string in this example would be set to AFFINITY_BLOB.
+ * string in this example would be set to SCALAR.
*/
static int
codeAllEqualityTerms(Parse * pParse, /* Parsing context */
Still some comments and variables contain ‘affinity’ term.
We can either accept it as a synonym to type, or fix
it within only code-style-fix commit.
Whole patch:
commit 53561119ee99de9d5a781c5e83c49374634c4b97
Author: Nikita Pettik <korablev@tarantool.org>
Date: Sun Dec 23 18:10:11 2018 +0200
sql: clean-up affinity from SQL source code
Replace remains of affinity usage in SQL parser, query optimizer and
VDBE. Don't add affinity to field definition when table is encoded into
msgpack. Remove field type <-> affinity converters, since now we can
operate directly on field type.
Part of #3698
diff --git a/src/box/sql.c b/src/box/sql.c
index a498cd8fe..8e2f5b6c7 100644
--- a/src/box/sql.c
+++ b/src/box/sql.c
@@ -997,7 +997,7 @@ sql_encode_table(struct region *region, struct Table *table, uint32_t *size)
uint32_t cid = def->fields[i].coll_id;
struct field_def *field = &def->fields[i];
const char *default_str = field->default_value;
- int base_len = 5;
+ int base_len = 4;
if (cid != COLL_NONE)
base_len += 1;
if (default_str != NULL)
@@ -1009,10 +1009,6 @@ sql_encode_table(struct region *region, struct Table *table, uint32_t *size)
assert(def->fields[i].is_nullable ==
action_is_nullable(def->fields[i].nullable_action));
mpstream_encode_str(&stream, field_type_strs[field->type]);
- mpstream_encode_str(&stream, "affinity");
- enum affinity_type aff =
- sql_field_type_to_affinity(def->fields[i].type);
- mpstream_encode_uint(&stream, aff);
mpstream_encode_str(&stream, "is_nullable");
mpstream_encode_bool(&stream, def->fields[i].is_nullable);
mpstream_encode_str(&stream, "nullable_action");
diff --git a/src/box/sql/build.c b/src/box/sql/build.c
index a0a8334f7..9eaa4300a 100644
--- a/src/box/sql/build.c
+++ b/src/box/sql/build.c
@@ -485,60 +485,6 @@ sql_field_retrieve(Parse *parser, Table *table, uint32_t id)
return field;
}
-enum field_type
-sql_affinity_to_field_type(enum affinity_type affinity)
-{
- switch (affinity) {
- case AFFINITY_INTEGER:
- return FIELD_TYPE_INTEGER;
- case AFFINITY_REAL:
- return FIELD_TYPE_NUMBER;
- case AFFINITY_TEXT:
- return FIELD_TYPE_STRING;
- case AFFINITY_BLOB:
- return FIELD_TYPE_SCALAR;
- case AFFINITY_UNDEFINED:
- return FIELD_TYPE_ANY;
- default:
- unreachable();
- }
-}
-
-enum affinity_type
-sql_field_type_to_affinity(enum field_type field_type)
-{
- switch (field_type) {
- case FIELD_TYPE_INTEGER:
- case FIELD_TYPE_UNSIGNED:
- return AFFINITY_INTEGER;
- case FIELD_TYPE_NUMBER:
- return AFFINITY_REAL;
- case FIELD_TYPE_STRING:
- return AFFINITY_TEXT;
- case FIELD_TYPE_SCALAR:
- return AFFINITY_BLOB;
- case FIELD_TYPE_ANY:
- return AFFINITY_UNDEFINED;
- default:
- unreachable();
- }
-}
-
-enum field_type *
-sql_affinity_str_to_field_type_str(const char *affinity_str)
-{
- if (affinity_str == NULL)
- return NULL;
- size_t len = strlen(affinity_str) + 1;
- size_t sz = len * sizeof(enum field_type);
- enum field_type *types =
- (enum field_type *) sqlite3DbMallocRaw(sql_get(), sz);
- for (uint32_t i = 0; i < len - 1; ++i)
- types[i] = sql_affinity_to_field_type(affinity_str[i]);
- types[len - 1] = field_type_MAX;
- return types;
-}
-
/*
* Add a new column to the table currently being constructed.
*
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index d3a8644ce..b0a595b97 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -107,6 +107,19 @@ sql_expr_type(struct Expr *pExpr)
return pExpr->type;
}
+enum field_type *
+field_type_sequence_dup(struct Parse *parse, enum field_type *types,
+ uint32_t len)
+{
+ uint32_t sz = (len + 1) * sizeof(enum field_type);
+ enum field_type *ret_types = sqlite3DbMallocRaw(parse->db, sz);
+ if (ret_types == NULL)
+ return NULL;
+ memcpy(ret_types, types, sz);
+ ret_types[len] = field_type_MAX;
+ return ret_types;
+}
+
/*
* Set the collating sequence for expression pExpr to be the collating
* sequence named by pToken. Return a pointer to a new Expr node that
@@ -308,8 +321,7 @@ binaryCompareP5(Expr * pExpr1, Expr * pExpr2, int jumpIfNull)
{
enum field_type lhs = sql_expr_type(pExpr2);
enum field_type rhs = sql_expr_type(pExpr1);
- u8 type_mask = sql_field_type_to_affinity(sql_type_result(rhs, lhs)) |
- (u8) jumpIfNull;
+ u8 type_mask = sql_type_result(rhs, lhs) | (u8) jumpIfNull;
return type_mask;
}
@@ -2574,16 +2586,16 @@ sqlite3FindInIndex(Parse * pParse, /* Parsing context */
* It is the responsibility of the caller to ensure that the returned
* string is eventually freed using sqlite3DbFree().
*/
-static char *
+static enum field_type *
exprINAffinity(Parse * pParse, Expr * pExpr)
{
Expr *pLeft = pExpr->pLeft;
int nVal = sqlite3ExprVectorSize(pLeft);
Select *pSelect = (pExpr->flags & EP_xIsSelect) ? pExpr->x.pSelect : 0;
- char *zRet;
assert(pExpr->op == TK_IN);
- zRet = sqlite3DbMallocZero(pParse->db, nVal + 1);
+ uint32_t sz = (nVal + 1) * sizeof(enum field_type);
+ enum field_type *zRet = sqlite3DbMallocZero(pParse->db, sz);
if (zRet) {
int i;
for (i = 0; i < nVal; i++) {
@@ -2592,12 +2604,14 @@ exprINAffinity(Parse * pParse, Expr * pExpr)
if (pSelect) {
struct Expr *e = pSelect->pEList->a[i].pExpr;
enum field_type rhs = sql_expr_type(e);
- zRet[i] = sql_field_type_to_affinity(sql_type_result(rhs, lhs));
+ zRet[i] = sql_type_result(rhs, lhs);
} else {
- zRet[i] = sql_field_type_to_affinity(lhs);
+ if (lhs == FIELD_TYPE_ANY)
+ lhs = FIELD_TYPE_SCALAR;
+ zRet[i] = lhs;
}
}
- zRet[nVal] = '\0';
+ zRet[nVal] = field_type_MAX;
}
return zRet;
}
@@ -2969,7 +2983,6 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
int rLhsOrig; /* LHS values prior to reordering by aiMap[] */
Vdbe *v; /* Statement under construction */
int *aiMap = 0; /* Map from vector field to index column */
- char *zAff = 0; /* Affinity string for comparisons */
int nVector; /* Size of vectors for this IN operator */
int iDummy; /* Dummy parameter to exprCodeVector() */
Expr *pLeft; /* The LHS of the IN operator */
@@ -2983,7 +2996,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
pLeft = pExpr->pLeft;
if (sqlite3ExprCheckIN(pParse, pExpr))
return;
- zAff = exprINAffinity(pParse, pExpr);
+ /* Type sequence for comparisons. */
+ enum field_type *zAff = exprINAffinity(pParse, pExpr);
nVector = sqlite3ExprVectorSize(pExpr->pLeft);
aiMap =
(int *)sqlite3DbMallocZero(pParse->db,
@@ -3129,9 +3143,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
- types[nVector] = field_type_MAX;
- sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
+ zAff[nVector] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char*)zAff,
P4_DYNAMIC);
if (destIfFalse == destIfNull) {
/* Combine Step 3 and Step 5 into a single opcode */
@@ -3213,7 +3226,9 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
if (rLhs != rLhsOrig)
sqlite3ReleaseTempReg(pParse, rLhs);
sqlite3ExprCachePop(pParse);
+ sqlite3DbFree(pParse->db, aiMap);
VdbeComment((v, "end IN expr"));
+ return;
sqlite3ExprCodeIN_oom_error:
sqlite3DbFree(pParse->db, aiMap);
sqlite3DbFree(pParse->db, zAff);
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index a02a74e88..a382f413f 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -41,33 +41,6 @@
#include "bit/bit.h"
#include "box/box.h"
-char *
-sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
- struct index_def *idx_def)
-{
- uint32_t column_count = idx_def->key_def->part_count;
- char *aff = (char *)sqlite3DbMallocRaw(db, column_count + 1);
- if (aff == NULL)
- return NULL;
- /*
- * Table may occasionally come from non-SQL API, so lets
- * gentle process this case by setting default affinity
- * for it.
- */
- if (space_def->fields == NULL) {
- memset(aff, AFFINITY_BLOB, column_count);
- } else {
- for (uint32_t i = 0; i < column_count; i++) {
- aff[i] = sql_space_index_part_affinity(space_def,
- idx_def, i);
- if (aff[i] == AFFINITY_UNDEFINED)
- aff[i] = AFFINITY_BLOB;
- }
- }
- aff[column_count] = '\0';
- return aff;
-}
-
enum field_type *
sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def)
{
@@ -1251,12 +1224,12 @@ xferOptimization(Parse * pParse, /* Parser context */
if (dest->def->field_count != src->def->field_count)
return 0;
for (i = 0; i < (int)dest->def->field_count; i++) {
- enum affinity_type dest_affinity =
- dest->def->fields[i].affinity;
- enum affinity_type src_affinity =
- src->def->fields[i].affinity;
- /* Affinity must be the same on all columns. */
- if (dest_affinity != src_affinity)
+ enum field_type dest_type =
+ dest->def->fields[i].type;
+ enum field_type src_type =
+ src->def->fields[i].type;
+ /* Type must be the same on all columns. */
+ if (dest_type != src_type)
return 0;
uint32_t id;
if (sql_column_collation(dest->def, i, &id) !=
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index 8a8ccc68c..92b563e22 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -1200,11 +1200,10 @@ selectInnerLoop(Parse * pParse, /* The parser context */
regOrig, nResultCol, nPrefixReg);
} else {
int r1 = sqlite3GetTempReg(pParse);
- assert(sqlite3Strlen30(pDest->zAffSdst) ==
- (unsigned int)nResultCol);
enum field_type *types =
- sql_affinity_str_to_field_type_str(
- pDest->zAffSdst);
+ field_type_sequence_dup(pParse,
+ pDest->zAffSdst,
+ nResultCol);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
nResultCol, r1, (char *)types,
P4_DYNAMIC);
@@ -1626,11 +1625,9 @@ generateSortTail(Parse * pParse, /* Parsing context */
break;
}
case SRT_Set:{
- assert((unsigned int)nColumn ==
- sqlite3Strlen30(pDest->zAffSdst));
-
enum field_type *types =
- sql_affinity_str_to_field_type_str(pDest->zAffSdst);
+ field_type_sequence_dup(pParse, pDest->zAffSdst,
+ nColumn);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
regTupleid, (char *)types,
P4_DYNAMIC);
@@ -1975,7 +1972,6 @@ sqlite3SelectAddColumnTypeAndCollation(Parse * pParse, /* Parsing contexts */
enum field_type type = sql_expr_type(p);
if (type == FIELD_TYPE_ANY)
type = FIELD_TYPE_SCALAR;
- pTab->def->fields[i].affinity = sql_field_type_to_affinity(type);
pTab->def->fields[i].type = type;
bool is_found;
uint32_t coll_id;
@@ -3069,7 +3065,8 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
enum field_type *types =
- sql_affinity_str_to_field_type_str(dest->zAffSdst);
+ field_type_sequence_dup(parse, dest->zAffSdst,
+ in->nSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
in->nSdst, r1, (char *)types,
P4_DYNAMIC);
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 209e7bf0f..822cc40df 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1792,12 +1792,6 @@ struct Savepoint {
(X) == FIELD_TYPE_NUMBER || \
(X) == FIELD_TYPE_UNSIGNED)
-/*
- * The AFFINITY_MASK values masks off the significant bits of an
- * affinity value.
- */
-#define AFFINITY_MASK 0x47
-
/*
* Additional bit values that can be ORed with an affinity without
* changing the affinity.
@@ -1807,7 +1801,7 @@ struct Savepoint {
* operator is NULL. It is added to certain comparison operators to
* prove that the operands are always NOT NULL.
*/
-#define SQLITE_KEEPNULL 0x08 /* Used by vector == or <> */
+#define SQLITE_KEEPNULL 0x40 /* Used by vector == or <> */
#define SQLITE_JUMPIFNULL 0x10 /* jumps if either operand is NULL */
#define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
#define SQLITE_NULLEQ 0x80 /* NULL=NULL */
@@ -2615,7 +2609,8 @@ struct Select {
*/
struct SelectDest {
u8 eDest; /* How to dispose of the results. On of SRT_* above. */
- char *zAffSdst; /* Affinity used when eDest==SRT_Set */
+ /* Affinity used when eDest==SRT_Set */
+ enum field_type *zAffSdst;
int iSDParm; /* A parameter used by the eDest disposal method */
/** Register containing ephemeral's space pointer. */
int reg_eph;
@@ -3431,19 +3426,6 @@ sql_create_view(struct Parse *parse_context, struct Token *begin,
struct Token *name, struct ExprList *aliases,
struct Select *select, bool if_exists);
-/**
- * Helper to convert SQLite affinity to corresponding
- * Tarantool field type.
- **/
-enum field_type
-sql_affinity_to_field_type(enum affinity_type affinity);
-
-enum affinity_type
-sql_field_type_to_affinity(enum field_type field_type);
-
-enum field_type *
-sql_affinity_str_to_field_type_str(const char *affinity_str);
-
/**
* Compile view, i.e. create struct Select from
* 'CREATE VIEW...' string, and assign cursors to each table from
@@ -4225,33 +4207,6 @@ int sqlite3VarintLen(u64 v);
#define getVarint sqlite3GetVarint
#define putVarint sqlite3PutVarint
-/**
- * Return a pointer to the column affinity string associated with
- * given index. A column affinity string has one character for
- * each column in the table, according to the affinity of the
- * column:
- *
- * Character Column affinity
- * ------------------------------
- * 'A' BLOB
- * 'B' TEXT
- * 'C' NUMERIC
- * 'D' INTEGER
- * 'F' REAL
- *
- * Memory for the buffer containing the column index affinity string
- * is allocated on heap.
- *
- * @param db Database handle.
- * @param space_def Definition of space index belongs to.
- * @param idx_def Definition of index from which affinity
- * to be extracted.
- * @retval Allocated affinity string, or NULL on OOM.
- */
-char *
-sql_space_index_affinity_str(struct sqlite3 *db, struct space_def *space_def,
- struct index_def *idx_def);
-
/** Return string consisting of fields types of given index. */
enum field_type *
sql_index_type_str(struct sqlite3 *db, const struct index_def *idx_def);
@@ -4292,6 +4247,13 @@ sql_expr_cmp_type_is_compatible(struct Expr *expr, enum field_type idx_type);
enum field_type
sql_expr_type(Expr *pExpr);
+/**
+ * This function duplicates first @len entries of types array
+ * and terminates new array with field_type_MAX member.
+ */
+enum field_type *
+field_type_sequence_dup(struct Parse *parse, enum field_type *types,
+ uint32_t len);
/**
* Convert z to a 64-bit signed integer. z must be decimal. This
@@ -4641,19 +4603,6 @@ int
sql_stat4_column(struct sqlite3 *db, const char *record, uint32_t col_num,
sqlite3_value **res);
-/**
- * Return the affinity for a single column of an index.
- *
- * @param def Definition of space @idx belongs to.
- * @param idx Index to be investigated.
- * @param partno Affinity of this part to be returned.
- *
- * @retval Affinity of @partno index part.
- */
-enum affinity_type
-sql_space_index_part_affinity(struct space_def *def, struct index_def *idx,
- uint32_t partno);
-
/*
* The interface to the LEMON-generated parser
*/
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 61d73b676..c3f596d4f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2022,14 +2022,6 @@ case OP_Cast: { /* in1 */
* jump to address P2. Or if the SQLITE_STOREP2 flag is set in P5, then
* store the result of comparison in register P2.
*
- * The AFFINITY_MASK portion of P5 must be an affinity character -
- * AFFINITY_TEXT, AFFINITY_INTEGER, and so forth. An attempt is made
- * to coerce both inputs according to this affinity before the
- * comparison is made. If the AFFINITY_MASK is 0x00, then numeric
- * affinity is used. Note that the affinity conversions are stored
- * back into the input registers P1 and P3. So this opcode can cause
- * persistent changes to registers P1 and P3.
- *
* Once any conversions have taken place, and neither value is NULL,
* the values are compared. If both values are blobs then memcmp() is
* used to determine the results of the comparison. If both values
@@ -2045,6 +2037,10 @@ case OP_Cast: { /* in1 */
* of comparison is true. If either operand is NULL then the result is false.
* If neither operand is NULL the result is the same as it would be if
* the SQLITE_NULLEQ flag were omitted from P5.
+ * P5 also can contain type to be applied to operands. Note that
+ * the type conversions are stored back into the input registers
+ * P1 and P3. So this opcode can cause persistent changes to
+ * registers P1 and P3.
*
* If both SQLITE_STOREP2 and SQLITE_KEEPNULL flags are set then the
* content of r[P2] is only changed if the new value is NULL or 0 (false).
@@ -2072,14 +2068,6 @@ case OP_Cast: { /* in1 */
* reg(P3) is NULL then the take the jump. If the SQLITE_JUMPIFNULL
* bit is clear then fall through if either operand is NULL.
*
- * The AFFINITY_MASK portion of P5 must be an affinity character -
- * AFFINITY_TEXT, AFFINITY_INTEGER, and so forth. An attempt is made
- * to coerce both inputs according to this affinity before the
- * comparison is made. If the AFFINITY_MASK is 0x00, then numeric
- * affinity is used. Note that the affinity conversions are stored
- * back into the input registers P1 and P3. So this opcode can cause
- * persistent changes to registers P1 and P3.
- *
* Once any conversions have taken place, and neither value is NULL,
* the values are compared. If both values are blobs then memcmp() is
* used to determine the results of the comparison. If both values
@@ -2163,9 +2151,16 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
break;
}
} else {
- /* Neither operand is NULL. Do a comparison. */
- affinity = pOp->p5 & AFFINITY_MASK;
- if (affinity>=AFFINITY_INTEGER) {
+ /*
+ * Neither operand is NULL. Do a comparison.
+ * 15 is 1111 in a binary form.
+ * Since all existing types can be fitted in 4 bits
+ * (field_type_MAX == 10), it is enough to 'and'
+ * p5 with this constant. Node that all other flags
+ * that can be stored in p5 are >= 16.
+ */
+ affinity = pOp->p5 & 15;
+ if (sql_type_is_numeric(affinity)) {
if ((flags1 | flags3)&MEM_Str) {
if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
applyNumericAffinity(pIn1,0);
@@ -2193,7 +2188,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- } else if (affinity==AFFINITY_TEXT) {
+ } else if (affinity == FIELD_TYPE_STRING) {
if ((flags1 & MEM_Str)==0 && (flags1 & (MEM_Int|MEM_Real))!=0) {
testcase( pIn1->flags & MEM_Int);
testcase( pIn1->flags & MEM_Real);
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 08eaddbfc..dbde4d832 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -3492,8 +3492,7 @@ sqlite3VdbeDb(Vdbe * v)
/*
* Return a pointer to an sqlite3_value structure containing the value bound
* parameter iVar of VM v. Except, if the value is an SQL NULL, return
- * 0 instead. Unless it is NULL, apply affinity aff (one of the AFFINITY_*
- * constants) to the value before returning it.
+ * 0 instead. Unless it is NULL, apply type to the value before returning it.
*
* The returned value must be freed by the caller using sqlite3ValueFree().
*/
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 219009ef8..6b3383bec 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -699,7 +699,6 @@ termCanDriveIndex(WhereTerm * pTerm, /* WHERE clause term to check */
return 0;
if (pTerm->u.leftColumn < 0)
return 0;
- aff = pSrc->pTab->aCol[pTerm->u.leftColumn].affinity;
if (!sql_expr_cmp_type_is_compatible(pTerm->pExpr, aff))
return 0;
return 1;
@@ -1137,15 +1136,6 @@ whereRangeAdjust(WhereTerm * pTerm, LogEst nNew)
return nRet;
}
-enum affinity_type
-sql_space_index_part_affinity(struct space_def *def, struct index_def *idx,
- uint32_t partno)
-{
- assert(partno < idx->key_def->part_count);
- uint32_t fieldno = idx->key_def->parts[partno].fieldno;
- return def->fields[fieldno].affinity;
-}
-
/*
* This function is called to estimate the number of rows visited by a
* range-scan on a skip-scan index. For example:
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index 33b860f36..01c1a2c6c 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -367,15 +367,15 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
* Code an OP_ApplyType opcode to apply the column type string types
* to the n registers starting at base.
*
- * As an optimization, AFFINITY_BLOB entries (which are no-ops) at the
+ * As an optimization, SCALAR entries (which are no-ops) at the
* beginning and end of zAff are ignored. If all entries in zAff are
- * AFFINITY_BLOB, then no code gets generated.
+ * SCALAR, then no code gets generated.
*
* This routine makes its own copy of zAff so that the caller is free
* to modify zAff after this routine returns.
*/
static void
-codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
+codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
{
Vdbe *v = pParse->pVdbe;
if (zAff == 0) {
@@ -384,23 +384,23 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
}
assert(v != 0);
- /* Adjust base and n to skip over AFFINITY_BLOB entries at the beginning
- * and end of the affinity string.
+ /*
+ * Adjust base and n to skip over SCALAR entries at the
+ * beginning and end of the type sequence.
*/
- while (n > 0 && zAff[0] == AFFINITY_BLOB) {
+ while (n > 0 && zAff[0] == FIELD_TYPE_SCALAR) {
n--;
base++;
zAff++;
}
- while (n > 1 && zAff[n - 1] == AFFINITY_BLOB) {
+ while (n > 1 && zAff[n - 1] == FIELD_TYPE_SCALAR) {
n--;
}
if (n > 0) {
- enum field_type *types =
- sql_affinity_str_to_field_type_str(zAff);
- types[n] = field_type_MAX;
- sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *)types,
+ enum field_type *types = field_type_sequence_dup(pParse, zAff,
+ n);
+ sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *) types,
P4_DYNAMIC);
sqlite3ExprCacheAffinityChange(pParse, base, n);
}
@@ -410,8 +410,8 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
* Expression pRight, which is the RHS of a comparison operation, is
* either a vector of n elements or, if n==1, a scalar expression.
* Before the comparison operation, affinity zAff is to be applied
- * to the pRight values. This function modifies characters within the
- * affinity string to AFFINITY_BLOB if either:
+ * to the pRight values. This function modifies entries within the
+ * field sequence to SCALAR if either:
*
* * the comparison will be performed with no affinity, or
* * the affinity change in zAff is guaranteed not to change the value.
@@ -419,16 +419,15 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
static void
updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
int n, /* Number of vector elements in comparison */
- char *zAff) /* Affinity string to modify */
+ enum field_type *zAff) /* Affinity string to modify */
{
int i;
for (i = 0; i < n; i++) {
- enum field_type type = sql_affinity_to_field_type(zAff[i]);
Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
enum field_type expr_type = sql_expr_type(p);
- if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
- sql_expr_needs_no_type_change(p, type)) {
- zAff[i] = AFFINITY_BLOB;
+ if (sql_type_result(expr_type, zAff[i]) == FIELD_TYPE_SCALAR ||
+ sql_expr_needs_no_type_change(p, zAff[i])) {
+ zAff[i] = FIELD_TYPE_SCALAR;
}
}
}
@@ -671,7 +670,7 @@ codeEqualityTerm(Parse * pParse, /* The parsing context */
* copy of the column affinity string of the index allocated using
* sqlite3DbMalloc(). Except, entries in the copy of the string associated
* with equality constraints that use BLOB or NONE affinity are set to
- * AFFINITY_BLOB. This is to deal with SQL such as the following:
+ * SCALAR. This is to deal with SQL such as the following:
*
* CREATE TABLE t1(a TEXT PRIMARY KEY, b);
* SELECT ... FROM t1 AS t2, t1 WHERE t1.a = t2.b;
@@ -680,14 +679,14 @@ codeEqualityTerm(Parse * pParse, /* The parsing context */
* the right hand side of the equality constraint (t2.b) has BLOB/NONE affinity,
* no conversion should be attempted before using a t2.b value as part of
* a key to search the index. Hence the first byte in the returned affinity
- * string in this example would be set to AFFINITY_BLOB.
+ * string in this example would be set to SCALAR.
*/
static int
codeAllEqualityTerms(Parse * pParse, /* Parsing context */
WhereLevel * pLevel, /* Which nested loop of the FROM we are coding */
int bRev, /* Reverse the order of IN operators */
int nExtraReg, /* Number of extra registers to allocate */
- char **pzAff) /* OUT: Set to point to affinity string */
+ enum field_type **pzAff) /* OUT: Set to point to affinity string */
{
u16 nEq; /* The number of == or IN constraints to code */
u16 nSkip; /* Number of left-most columns to skip */
@@ -711,11 +710,7 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
nReg = pLoop->nEq + nExtraReg;
pParse->nMem += nReg;
-
- struct space *space = space_by_id(idx_def->space_id);
- assert(space != NULL);
- char *zAff = sql_space_index_affinity_str(pParse->db, space->def,
- idx_def);
+ enum field_type *zAff = sql_index_type_str(pParse->db, idx_def);
assert(zAff != 0 || pParse->db->mallocFailed);
if (nSkip) {
@@ -741,7 +736,6 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
/* Evaluate the equality constraints
*/
- assert(zAff == 0 || (int)strlen(zAff) >= nEq);
for (j = nSkip; j < nEq; j++) {
int r1;
pTerm = pLoop->aLTerm[j];
@@ -769,7 +763,7 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
* affinity of the comparison has been applied to the value.
*/
if (zAff)
- zAff[j] = AFFINITY_BLOB;
+ zAff[j] = FIELD_TYPE_SCALAR;
}
} else if ((pTerm->eOperator & WO_ISNULL) == 0) {
Expr *pRight = pTerm->pExpr->pRight;
@@ -781,14 +775,13 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
if (zAff) {
enum field_type type =
sql_expr_type(pRight);
- enum field_type idx_type =
- sql_affinity_to_field_type(zAff[j]);
+ enum field_type idx_type = zAff[j];
if (sql_type_result(type, idx_type) ==
FIELD_TYPE_SCALAR) {
- zAff[j] = AFFINITY_BLOB;
+ zAff[j] = FIELD_TYPE_SCALAR;
}
if (sql_expr_needs_no_type_change(pRight, idx_type))
- zAff[j] = AFFINITY_BLOB;
+ zAff[j] = FIELD_TYPE_SCALAR;
}
}
}
@@ -1001,8 +994,8 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
int iIdxCur; /* The VDBE cursor for the index */
int nExtraReg = 0; /* Number of extra registers needed */
int op; /* Instruction opcode */
- char *zStartAff; /* Affinity for start of range constraint */
- char *zEndAff = 0; /* Affinity for end of range constraint */
+ enum field_type *zStartAff; /* Affinity for start of range constraint */
+ enum field_type *zEndAff = NULL; /* Affinity for end of range constraint */
u8 bSeekPastNull = 0; /* True to seek past initial nulls */
u8 bStopAtNull = 0; /* Add condition to terminate at NULLs */
int force_integer_reg = -1; /* If non-negative: number of
@@ -1117,9 +1110,13 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
regBase =
codeAllEqualityTerms(pParse, pLevel, bRev, nExtraReg,
&zStartAff);
- assert(zStartAff == 0 || sqlite3Strlen30(zStartAff) >= nEq);
- if (zStartAff && nTop) {
- zEndAff = sqlite3DbStrDup(db, &zStartAff[nEq]);
+ if (zStartAff != NULL && nTop) {
+ uint32_t len = 0;
+ for (enum field_type *tmp = &zStartAff[nEq];
+ *tmp != field_type_MAX; tmp++, len++);
+ uint32_t sz = len * sizeof(enum field_type);
+ zEndAff = sqlite3DbMallocRaw(db, sz);
+ memcpy(zEndAff, &zStartAff[nEq], sz);
}
addrNxt = pLevel->addrNxt;
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index 6ffd5ae84..96838f4b2 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -284,7 +284,8 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
Vdbe *pReprepare = pParse->pReprepare;
int iCol = pRight->iColumn;
pVal =
- sqlite3VdbeGetBoundValue(pReprepare, iCol, AFFINITY_BLOB);
+ sqlite3VdbeGetBoundValue(pReprepare, iCol,
+ FIELD_TYPE_SCALAR);
if (pVal && sqlite3_value_type(pVal) == SQLITE_TEXT) {
z = (char *)sqlite3_value_text(pVal);
}
diff --git a/test/sql/types.result b/test/sql/types.result
index 915a6341a..df4dc151e 100644
--- a/test/sql/types.result
+++ b/test/sql/types.result
@@ -33,20 +33,19 @@ box.sql.execute("CREATE TABLE t1 (id TEXT PRIMARY KEY, a REAL, b INT, c TEXT, d
...
box.space.T1:format()
---
-- [{'affinity': 66, 'type': 'string', 'nullable_action': 'abort', 'name': 'ID', 'is_nullable': false},
- {'affinity': 69, 'type': 'number', 'nullable_action': 'none', 'name': 'A', 'is_nullable': true},
- {'affinity': 68, 'type': 'integer', 'nullable_action': 'none', 'name': 'B', 'is_nullable': true},
- {'affinity': 66, 'type': 'string', 'nullable_action': 'none', 'name': 'C', 'is_nullable': true},
- {'affinity': 65, 'type': 'scalar', 'nullable_action': 'none', 'name': 'D', 'is_nullable': true}]
+- [{'type': 'string', 'nullable_action': 'abort', 'name': 'ID', 'is_nullable': false},
+ {'type': 'number', 'nullable_action': 'none', 'name': 'A', 'is_nullable': true},
+ {'type': 'integer', 'nullable_action': 'none', 'name': 'B', 'is_nullable': true},
+ {'type': 'string', 'nullable_action': 'none', 'name': 'C', 'is_nullable': true},
+ {'type': 'scalar', 'nullable_action': 'none', 'name': 'D', 'is_nullable': true}]
...
box.sql.execute("CREATE VIEW v1 AS SELECT b + a, b - a FROM t1;")
---
...
box.space.V1:format()
---
-- [{'affinity': 69, 'type': 'number', 'nullable_action': 'none', 'name': 'b + a',
- 'is_nullable': true}, {'affinity': 69, 'type': 'number', 'nullable_action': 'none',
- 'name': 'b - a', 'is_nullable': true}]
+- [{'type': 'number', 'nullable_action': 'none', 'name': 'b + a', 'is_nullable': true},
+ {'type': 'number', 'nullable_action': 'none', 'name': 'b - a', 'is_nullable': true}]
...
-- gh-2494: index's part also features correct declared type.
--
diff --git a/test/sql/upgrade.result b/test/sql/upgrade.result
index 79c7eb245..02ab9b42b 100644
--- a/test/sql/upgrade.result
+++ b/test/sql/upgrade.result
@@ -80,12 +80,12 @@ box.sql.execute("CREATE TRIGGER t2t AFTER INSERT ON t BEGIN INSERT INTO t_out VA
...
box.space._space.index['name']:get('T')
---
-- [513, 1, 'T', 'memtx', 1, {}, [{'affinity': 68, 'type': 'integer', 'nullable_action': 'abort',
- 'name': 'X', 'is_nullable': false}]]
+- [513, 1, 'T', 'memtx', 1, {}, [{'type': 'integer', 'nullable_action': 'abort', 'name': 'X',
+ 'is_nullable': false}]]
...
box.space._space.index['name']:get('T_OUT')
---
-- [514, 1, 'T_OUT', 'memtx', 1, {}, [{'affinity': 68, 'type': 'integer', 'nullable_action': 'abort',
+- [514, 1, 'T_OUT', 'memtx', 1, {}, [{'type': 'integer', 'nullable_action': 'abort',
'name': 'X', 'is_nullable': false}]]
...
t1t = box.space._trigger:get('T1T')
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2019-01-16 14:26 ` n.pettik
@ 2019-01-22 15:41 ` Vladislav Shpilevoy
2019-01-28 16:40 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-22 15:41 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
> Still some comments and variables contain ‘affinity’ term.
> We can either accept it as a synonym to type, or fix
> it within only code-style-fix commit.
I think, we should rename. As soon as possible. Otherwise we
will have 'affinity' in our code base for ages. Together with
comments, dead code, already changed in this patch code, and
the code you remove in the next commit my text editor found
199 'affinity' usages. It is not too much. Also, I am almost
sure, that you will find some bugs during the renaming process.
> commit 53561119ee99de9d5a781c5e83c49374634c4b97
> Author: Nikita Pettik <korablev@tarantool.org>
> Date: Sun Dec 23 18:10:11 2018 +0200
>
> sql: clean-up affinity from SQL source code
>
> Replace remains of affinity usage in SQL parser, query optimizer and
> VDBE. Don't add affinity to field definition when table is encoded into
> msgpack. Remove field type <-> affinity converters, since now we can
> operate directly on field type.
>
> Part of #3698
See 6 comments below.
>
> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
> index d3a8644ce..b0a595b97 100644
> --- a/src/box/sql/expr.c
> +++ b/src/box/sql/expr.c
> @@ -3213,7 +3226,9 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
> if (rLhs != rLhsOrig)
> sqlite3ReleaseTempReg(pParse, rLhs);
> sqlite3ExprCachePop(pParse);
> + sqlite3DbFree(pParse->db, aiMap);
> VdbeComment((v, "end IN expr"));
> + return;
1. What was wrong with the previous way of the function finalization?
I do not see any functional changes in this function, but logic of
freeing is different. Why? Because zAff should not be freed? Then please,
fix it in the previous commit. And to do not change logic of
"sqlite3ExprCodeIN_finished:" you can nullify zAff after passing it to
OP_ApplyType above.
> sqlite3ExprCodeIN_oom_error:
> sqlite3DbFree(pParse->db, aiMap);
> sqlite3DbFree(pParse->db, zAff);
> diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
> index 209e7bf0f..822cc40df 100644
> --- a/src/box/sql/sqliteInt.h
> +++ b/src/box/sql/sqliteInt.h
> @@ -1807,7 +1801,7 @@ struct Savepoint {
> * operator is NULL. It is added to certain comparison operators to
> * prove that the operands are always NOT NULL.
> */
> -#define SQLITE_KEEPNULL 0x08 /* Used by vector == or <> */
> +#define SQLITE_KEEPNULL 0x40 /* Used by vector == or <> */
> #define SQLITE_JUMPIFNULL 0x10 /* jumps if either operand is NULL */
> #define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
> #define SQLITE_NULLEQ 0x80 /* NULL=NULL */
2. Please, keep them sorted. Cuts the eye.
> @@ -2615,7 +2609,8 @@ struct Select {
> */
> struct SelectDest {
> u8 eDest; /* How to dispose of the results. On of SRT_* above. */
> - char *zAffSdst; /* Affinity used when eDest==SRT_Set */
> + /* Affinity used when eDest==SRT_Set */
> + enum field_type *zAffSdst;
3. Not affinity.
> int iSDParm; /* A parameter used by the eDest disposal method */
> /** Register containing ephemeral's space pointer. */
> int reg_eph;
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 61d73b676..c3f596d4f 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -2163,9 +2151,16 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
> break;
> }
> } else {
> - /* Neither operand is NULL. Do a comparison. */
> - affinity = pOp->p5 & AFFINITY_MASK;
> - if (affinity>=AFFINITY_INTEGER) {
> + /*
> + * Neither operand is NULL. Do a comparison.
> + * 15 is 1111 in a binary form.
> + * Since all existing types can be fitted in 4 bits
> + * (field_type_MAX == 10), it is enough to 'and'
> + * p5 with this constant. Node that all other flags
> + * that can be stored in p5 are >= 16.
> + */
> + affinity = pOp->p5 & 15;
4. Honestly, I did not expect it from you. Please, never use such
constants and assumptions about them in code. field_type_MAX = 10 can
be changed any day (for example, when we introduce decimal as a native
Tarantool data type, or decide to add datetime). In fact, it is already
not 10 - it is 9.
I propose you to either come up with a better solution or to use one of
my solutions:
* return AFFINITY_MASK as P5_FIELD_TYPE_MASK, and add a static assertion
that P5_FIELD_TYPE_MASK < (1 << bit_count(field_type)).
Or write in field_def.h something like this:
static_assert(bit_count(field_type) <= 4,
"size of enum field_type is used in "\
"VdbeOp.p5 to fit it into 4 bits");
To get bit_count during compilation you can specify it explicitly right
after field_type_MAX in enum field_type so as to change it always
together with field_type_MAX.
* split u16 VdbeOp.p5 into u8 VdbeOp.p6 and u8 VdbeOp.p5. But I am not
sure if all p5 flags fit into u8.
> + if (sql_type_is_numeric(affinity)) {
> if ((flags1 | flags3)&MEM_Str) {
> if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
> applyNumericAffinity(pIn1,0);
> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
> index 33b860f36..01c1a2c6c 100644
> --- a/src/box/sql/wherecode.c
> +++ b/src/box/sql/wherecode.c
> @@ -367,15 +367,15 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
> * Code an OP_ApplyType opcode to apply the column type string types
> * to the n registers starting at base.
> *
> - * As an optimization, AFFINITY_BLOB entries (which are no-ops) at the
> + * As an optimization, SCALAR entries (which are no-ops) at the
> * beginning and end of zAff are ignored. If all entries in zAff are
> - * AFFINITY_BLOB, then no code gets generated.
> + * SCALAR, then no code gets generated.
> *
> * This routine makes its own copy of zAff so that the caller is free
> * to modify zAff after this routine returns.
> */
> static void
> -codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
> +codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
5. Not affinity.
> {
> Vdbe *v = pParse->pVdbe;
> if (zAff == 0) {
> @@ -419,16 +419,15 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
> static void
> updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
> int n, /* Number of vector elements in comparison */
> - char *zAff) /* Affinity string to modify */
> + enum field_type *zAff) /* Affinity string to modify */
6. The same. In many other places too. Where you changed type and
usages of a variable, but kept its name.
> {
> int i;
> for (i = 0; i < n; i++) {
> - enum field_type type = sql_affinity_to_field_type(zAff[i]);
> Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
> enum field_type expr_type = sql_expr_type(p);
> - if (sql_type_result(expr_type, type) == FIELD_TYPE_SCALAR ||
> - sql_expr_needs_no_type_change(p, type)) {
> - zAff[i] = AFFINITY_BLOB;
> + if (sql_type_result(expr_type, zAff[i]) == FIELD_TYPE_SCALAR ||
> + sql_expr_needs_no_type_change(p, zAff[i])) {
> + zAff[i] = FIELD_TYPE_SCALAR;
> }
> }
> }
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2019-01-22 15:41 ` Vladislav Shpilevoy
@ 2019-01-28 16:40 ` n.pettik
2019-01-30 13:04 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-01-28 16:40 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> Still some comments and variables contain ‘affinity’ term.
>> We can either accept it as a synonym to type, or fix
>> it within only code-style-fix commit.
>
> I think, we should rename. As soon as possible. Otherwise we
> will have 'affinity' in our code base for ages. Together with
> comments, dead code, already changed in this patch code, and
> the code you remove in the next commit my text editor found
> 199 'affinity' usages. It is not too much. Also, I am almost
> sure, that you will find some bugs during the renaming process.
Ok, I’ve dealt with it. You may say that some renaming/refactoring
rather belongs to other patches, but lets consider this patch as a
follow-up containing code-style fixes. It seems to be quite complicated
to split these fixes to proper parts.
diff --git a/src/box/sql/delete.c b/src/box/sql/delete.c
index a6e7547f5..5a3be4862 100644
--- a/src/box/sql/delete.c
+++ b/src/box/sql/delete.c
@@ -587,7 +587,7 @@ sql_generate_index_key(struct Parse *parse, struct index *index, int cursor,
sqlite3ExprCodeGetColumnOfTable(v, space->def, cursor, tabl_col,
reg_base + j);
/*
- * If the column affinity is REAL but the number
+ * If the column type is NUMBER but the number
* is an integer, then it might be stored in the
* table as an integer (using a compact
* representation) then converted to REAL by an
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index ffb3f8bb8..c3fd70427 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -267,7 +267,7 @@ sql_type_result(enum field_type lhs, enum field_type rhs)
}
/**
- * pExpr is a comparison operator. Return the type affinity
+ * pExpr is a comparison operator. Return the type
* that should be applied to both operands prior to doing
* the comparison.
*/
@@ -2391,15 +2391,15 @@ sqlite3FindInIndex(Parse * pParse, /* Parsing context */
pTab = p->pSrc->a[0].pTab;
assert(v); /* sqlite3GetVdbe() has always been previously called */
- int affinity_ok = 1;
+ bool type_is_suitable = true;
int i;
- /* Check that the affinity that will be used to perform each
- * comparison is the same as the affinity of each column in table
+ /* Check that the type that will be used to perform each
+ * comparison is the same as the type of each column in table
* on the RHS of the IN operator. If it not, it is not possible to
* use any index of the RHS table.
*/
- for (i = 0; i < nExpr && affinity_ok; i++) {
+ for (i = 0; i < nExpr && type_is_suitable; i++) {
Expr *pLhs = sqlite3VectorFieldSubexpr(pX->pLeft, i);
int iCol = pEList->a[i].pExpr->iColumn;
/* RHS table */
@@ -2411,10 +2411,10 @@ sqlite3FindInIndex(Parse * pParse, /* Parsing context */
* of columns match.
*/
if (idx_type != lhs_type)
- affinity_ok = 0;
+ type_is_suitable = false;
}
- if (affinity_ok) {
+ if (type_is_suitable) {
/*
* Here we need real space since further
* it is used in cursor opening routine.
@@ -2531,7 +2531,7 @@ sqlite3FindInIndex(Parse * pParse, /* Parsing context */
sqlite3VdbeJumpHere(v, iAddr);
}
} /* End loop over indexes */
- } /* End if( affinity_ok ) */
+ }
}
/* End attempt to optimize using an index */
@@ -2580,14 +2580,14 @@ sqlite3FindInIndex(Parse * pParse, /* Parsing context */
/*
* Argument pExpr is an (?, ?...) IN(...) expression. This
- * function allocates and returns a nul-terminated string containing
- * the affinities to be used for each column of the comparison.
+ * function allocates and returns a terminated string containing
+ * the types to be used for each column of the comparison.
*
* It is the responsibility of the caller to ensure that the returned
* string is eventually freed using sqlite3DbFree().
*/
static enum field_type *
-exprINAffinity(Parse * pParse, Expr * pExpr)
+expr_in_type(Parse *pParse, Expr *pExpr)
{
Expr *pLeft = pExpr->pLeft;
int nVal = sqlite3ExprVectorSize(pLeft);
@@ -2606,8 +2606,6 @@ exprINAffinity(Parse * pParse, Expr * pExpr)
enum field_type rhs = sql_expr_type(e);
zRet[i] = sql_type_result(rhs, lhs);
} else {
- if (lhs == FIELD_TYPE_ANY)
- lhs = FIELD_TYPE_SCALAR;
zRet[i] = lhs;
}
}
@@ -2725,12 +2723,12 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
* SELECT or the <exprlist>.
*
* If the 'x' expression is a column value, or the SELECT...
- * statement returns a column value, then the affinity of that
+ * statement returns a column value, then the type of that
* column is used to build the index keys. If both 'x' and the
- * SELECT... statement are columns, then numeric affinity is used
- * if either column has NUMERIC or INTEGER affinity. If neither
- * 'x' nor the SELECT... statement are columns, then numeric affinity
- * is used.
+ * SELECT... statement are columns, then NUMBER type is used
+ * if either column has NUMBER or INTEGER type. If neither
+ * 'x' nor the SELECT... statement are columns,
+ * then NUMBER type is used.
*/
pExpr->iTable = pParse->nTab++;
pExpr->is_ephemeral = 1;
@@ -2760,8 +2758,8 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
int i;
sqlite3SelectDestInit(&dest, SRT_Set,
pExpr->iTable, reg_eph);
- dest.zAffSdst =
- exprINAffinity(pParse, pExpr);
+ dest.dest_type =
+ expr_in_type(pParse, pExpr);
assert((pExpr->iTable & 0x0000FFFF) ==
pExpr->iTable);
pSelect->iLimit = 0;
@@ -2770,12 +2768,12 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
if (sqlite3Select
(pParse, pSelect, &dest)) {
sqlite3DbFree(pParse->db,
- dest.zAffSdst);
+ dest.dest_type);
sql_key_info_unref(key_info);
return 0;
}
sqlite3DbFree(pParse->db,
- dest.zAffSdst);
+ dest.dest_type);
assert(pEList != 0);
assert(pEList->nExpr > 0);
for (i = 0; i < nVal; i++) {
@@ -2792,8 +2790,8 @@ sqlite3CodeSubselect(Parse * pParse, /* Parsing context */
*
* For each expression, build an index key from the evaluation and
* store it in the temporary table. If <expr> is a column, then use
- * that columns affinity when building index keys. If <expr> is not
- * a column, use numeric affinity.
+ * that columns types when building index keys. If <expr> is not
+ * a column, use NUMBER type.
*/
int i;
ExprList *pList = pExpr->x.pList;
@@ -2999,7 +2997,7 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
if (sqlite3ExprCheckIN(pParse, pExpr))
return;
/* Type sequence for comparisons. */
- enum field_type *zAff = exprINAffinity(pParse, pExpr);
+ enum field_type *zAff = expr_in_type(pParse, pExpr);
nVector = sqlite3ExprVectorSize(pExpr->pLeft);
aiMap =
(int *)sqlite3DbMallocZero(pParse->db,
@@ -3981,7 +3979,7 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
} else {
/*
* Otherwise, use first arg as
- * expression affinity.
+ * expression type.
*/
if (pFarg && pFarg->nExpr > 0)
pExpr->type = pFarg->a[0].pExpr->type;
@@ -4209,11 +4207,8 @@ sqlite3ExprCodeTarget(Parse * pParse, Expr * pExpr, int target)
pExpr->iColumn].name, target));
#ifndef SQLITE_OMIT_FLOATING_POINT
- /* If the column has REAL affinity, it may currently be stored as an
+ /* If the column has type NUMBER, it may currently be stored as an
* integer. Use OP_Realify to make sure it is really real.
- *
- * EVIDENCE-OF: R-60985-57662 SQLite will convert the value back to
- * floating point when extracting it from the record.
*/
if (pExpr->iColumn >= 0 && def->fields[
pExpr->iColumn].type == FIELD_TYPE_NUMBER) {
diff --git a/src/box/sql/fkey.c b/src/box/sql/fkey.c
index d5a728382..01fe1c84d 100644
--- a/src/box/sql/fkey.c
+++ b/src/box/sql/fkey.c
@@ -423,7 +423,7 @@ fkey_scan_children(struct Parse *parser, struct SrcList *src, struct Table *tab,
* <parent-key1> = <child-key1> AND <parent-key2> = <child-key2> ...
*
* The collation sequence used for the comparison should
- * be that of the parent key columns. The affinity of the
+ * be that of the parent key columns. The type of the
* parent key column should be applied to each child key
* value before the comparison takes place.
*/
@@ -783,7 +783,7 @@ fkey_action_trigger(struct Parse *pParse, struct Table *pTab, struct fkey *fkey,
* Create the expression "old.to_col = from_col".
* It is important that the "old.to_col" term is
* on the LHS of the = operator, so that the
- * affinity and collation sequence associated with
+ * type and collation sequence associated with
* the parent table are used for the comparison.
*/
struct Expr *to_col =
diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
index f4e5b1ec0..c69476117 100644
--- a/src/box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -603,7 +603,7 @@ sqlite3Insert(Parse * pParse, /* Parser context */
/* If this is an INSERT on a view with an INSTEAD OF INSERT trigger,
* do not attempt any conversions before assembling the record.
* If this is a real table, attempt conversions as required by the
- * table column affinities.
+ * table column types.
*/
if (!is_view)
sql_emit_table_types(v, pTab->def, regCols + 1);
diff --git a/src/box/sql/resolve.c b/src/box/sql/resolve.c
index 774a34fd7..336c7af7a 100644
--- a/src/box/sql/resolve.c
+++ b/src/box/sql/resolve.c
@@ -738,7 +738,7 @@ resolveExprStep(Walker * pWalker, Expr * pExpr)
}
pNC->ncFlags |= NC_AllowAgg;
}
- /* FIX ME: Compute pExpr->affinity based on the expected return
+ /* FIX ME: Compute pExpr->type based on the expected return
* type of the function
*/
return WRC_Prune;
diff --git a/src/box/sql/select.c b/src/box/sql/select.c
index d582a1f19..714c0c34a 100644
--- a/src/box/sql/select.c
+++ b/src/box/sql/select.c
@@ -134,7 +134,7 @@ sqlite3SelectDestInit(SelectDest * pDest, int eDest, int iParm, int reg_eph)
pDest->eDest = (u8) eDest;
pDest->iSDParm = iParm;
pDest->reg_eph = reg_eph;
- pDest->zAffSdst = 0;
+ pDest->dest_type = NULL;
pDest->iSdst = 0;
pDest->nSdst = 0;
}
@@ -1202,7 +1202,7 @@ selectInnerLoop(Parse * pParse, /* The parser context */
int r1 = sqlite3GetTempReg(pParse);
enum field_type *types =
field_type_sequence_dup(pParse,
- pDest->zAffSdst,
+ pDest->dest_type,
nResultCol);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regResult,
nResultCol, r1, (char *)types,
@@ -1626,7 +1626,7 @@ generateSortTail(Parse * pParse, /* Parsing context */
}
case SRT_Set:{
enum field_type *types =
- field_type_sequence_dup(pParse, pDest->zAffSdst,
+ field_type_sequence_dup(pParse, pDest->dest_type,
nColumn);
sqlite3VdbeAddOp4(v, OP_MakeRecord, regRow, nColumn,
regTupleid, (char *)types,
@@ -3055,7 +3055,7 @@ generateOutputSubroutine(struct Parse *parse, struct Select *p,
testcase(in->nSdst > 1);
r1 = sqlite3GetTempReg(parse);
enum field_type *types =
- field_type_sequence_dup(parse, dest->zAffSdst,
+ field_type_sequence_dup(parse, dest->dest_type,
in->nSdst);
sqlite3VdbeAddOp4(v, OP_MakeRecord, in->iSdst,
in->nSdst, r1, (char *)types,
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 260431d6d..88813b674 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1629,7 +1629,7 @@ struct sqlite3 {
#define SQLITE_MAGIC_ZOMBIE 0x64cffc7f /* Close with last statement close */
/**
- * SQL type definition. Now it is an alias to affinity, but in
+ * SQL type definition. Now it is an alias to type, but in
* future it will have some attributes like number of chars in
* VARCHAR(<number of chars>).
*/
@@ -1793,8 +1793,8 @@ struct Savepoint {
(X) == FIELD_TYPE_UNSIGNED)
/*
- * Additional bit values that can be ORed with an affinity without
- * changing the affinity.
+ * Additional bit values that can be ORed with an type without
+ * changing the type.
*
* The SQLITE_NOTNULL flag is a combination of NULLEQ and JUMPIFNULL.
* It causes an assert() to fire if either operand to a comparison
@@ -2550,7 +2550,7 @@ struct Select {
*
* SRT_Set The result must be a single column. Store each
* row of result as the key in table pDest->iSDParm.
- * Apply the affinity pDest->affSdst before storing
+ * Apply the type pDest->det_type before storing
* results. Used to implement "IN (SELECT ...)".
*
* SRT_EphemTab Create an temporary table pDest->iSDParm and store
@@ -2610,7 +2610,7 @@ struct Select {
struct SelectDest {
u8 eDest; /* How to dispose of the results. On of SRT_* above. */
/** Type used when eDest==SRT_Set */
- enum field_type *zAffSdst;
+ enum field_type *dest_type;
int iSDParm; /* A parameter used by the eDest disposal method */
/** Register containing ephemeral's space pointer. */
int reg_eph;
@@ -4371,7 +4371,7 @@ void sqlite3ValueFree(sqlite3_value *);
sqlite3_value *sqlite3ValueNew(sqlite3 *);
int sqlite3ValueFromExpr(sqlite3 *, Expr *, enum field_type type,
sqlite3_value **);
-void sqlite3ValueApplyAffinity(sqlite3_value *, enum field_type type);
+void sql_value_apply_type(sqlite3_value *val, enum field_type type);
#ifndef SQLITE_AMALGAMATION
extern const unsigned char sqlite3OpcodeProperty[];
extern const char sqlite3StrBINARY[];
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index df59f6a0a..c98fcfabd 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -289,7 +289,7 @@ allocateCursor(
* if there is an exact integer representation of the quantity.
*/
static int
-applyNumericAffinity(Mem *pRec, int bTryForInt)
+mem_apply_numeric_type(Mem *pRec, int bTryForInt)
{
double rValue;
i64 iValue;
@@ -301,7 +301,7 @@ applyNumericAffinity(Mem *pRec, int bTryForInt)
} else {
pRec->u.r = rValue;
pRec->flags |= MEM_Real;
- if (bTryForInt) sqlite3VdbeIntegerAffinity(pRec);
+ if (bTryForInt) mem_apply_integer_type(pRec);
}
return 0;
}
@@ -382,7 +382,7 @@ int sqlite3_value_numeric_type(sqlite3_value *pVal) {
int eType = sqlite3_value_type(pVal);
if (eType==SQLITE_TEXT) {
Mem *pMem = (Mem*)pVal;
- applyNumericAffinity(pMem, 0);
+ mem_apply_numeric_type(pMem, 0);
eType = sqlite3_value_type(pVal);
}
return eType;
@@ -393,7 +393,7 @@ int sqlite3_value_numeric_type(sqlite3_value *pVal) {
* not the internal Mem* type.
*/
void
-sqlite3ValueApplyAffinity(
+sql_value_apply_type(
sqlite3_value *pVal,
enum field_type type)
{
@@ -421,7 +421,7 @@ static u16 SQLITE_NOINLINE computeNumericType(Mem *pMem)
* Return the numeric type for pMem, either MEM_Int or MEM_Real or both or
* none.
*
- * Unlike applyNumericAffinity(), this routine does not modify pMem->flags.
+ * Unlike mem_apply_numeric_type(), this routine does not modify pMem->flags.
* But it does set pMem->u.r and pMem->u.i appropriately.
*/
static u16 numericType(Mem *pMem)
@@ -1681,7 +1681,7 @@ case OP_Remainder: { /* same as TK_REM, in1, in2, out3 */
pOut->u.r = rB;
MemSetTypeFlag(pOut, MEM_Real);
if (((type1|type2)&MEM_Real)==0 && !bIntint) {
- sqlite3VdbeIntegerAffinity(pOut);
+ mem_apply_integer_type(pOut);
}
#endif
}
@@ -2107,7 +2107,6 @@ case OP_Le: /* same as TK_LE, jump, in1, in3 */
case OP_Gt: /* same as TK_GT, jump, in1, in3 */
case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
int res, res2; /* Result of the comparison of pIn1 against pIn3 */
- char affinity; /* Affinity to use for comparison */
u32 flags1; /* Copy of initial value of pIn1->flags */
u32 flags3; /* Copy of initial value of pIn3->flags */
@@ -2160,16 +2159,16 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
* p5 with this constant. Node that all other flags
* that can be stored in p5 are >= 16.
*/
- affinity = pOp->p5 & 15;
- if (sql_type_is_numeric(affinity)) {
+ enum field_type type = pOp->p5 & 15;
+ if (sql_type_is_numeric(type)) {
if ((flags1 | flags3)&MEM_Str) {
if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
- applyNumericAffinity(pIn1,0);
+ mem_apply_numeric_type(pIn1, 0);
testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
flags3 = pIn3->flags;
}
if ((flags3 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
- if (applyNumericAffinity(pIn3,0) != 0) {
+ if (mem_apply_numeric_type(pIn3, 0) != 0) {
diag_set(ClientError,
ER_SQL_TYPE_MISMATCH,
sqlite3_value_text(pIn3),
@@ -2189,7 +2188,7 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
res = 0;
goto compare_op;
}
- } else if (affinity == FIELD_TYPE_STRING) {
+ } else if (type == FIELD_TYPE_STRING) {
if ((flags1 & MEM_Str)==0 && (flags1 & (MEM_Int|MEM_Real))!=0) {
testcase( pIn1->flags & MEM_Int);
testcase( pIn1->flags & MEM_Real);
@@ -2837,7 +2836,7 @@ case OP_ApplyType: {
* in an index. The OP_Column opcode can decode the record later.
*
* P4 may be a string that is P2 characters long. The nth character of the
- * string indicates the column affinity that should be used for the nth
+ * string indicates the column type that should be used for the nth
* field of the index key.
*
* If P4 is NULL then all index fields have type SCALAR.
@@ -2880,8 +2879,7 @@ case OP_MakeRecord: {
pOut = &aMem[pOp->p3];
memAboutToChange(p, pOut);
- /* Apply the requested affinity to all inputs
- */
+ /* Apply the requested types to all inputs */
assert(pData0<=pLast);
if (types != NULL) {
pRec = pData0;
@@ -3513,7 +3511,7 @@ case OP_SeekGT: { /* jump, in3 */
*/
pIn3 = &aMem[reg_ipk];
if ((pIn3->flags & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
- applyNumericAffinity(pIn3, 0);
+ mem_apply_numeric_type(pIn3, 0);
}
int64_t i;
if ((pIn3->flags & MEM_Int) == MEM_Int) {
diff --git a/src/box/sql/vdbeInt.h b/src/box/sql/vdbeInt.h
index 32b1b6c17..1720fd1c7 100644
--- a/src/box/sql/vdbeInt.h
+++ b/src/box/sql/vdbeInt.h
@@ -231,7 +231,6 @@ struct Mem {
#define MEM_Blob 0x0010 /* Value is a BLOB */
#define MEM_Bool 0x0020 /* Value is a bool */
#define MEM_Ptr 0x0040 /* Value is a generic pointer */
-#define MEM_AffMask 0x003f /* Mask of affinity bits */
#define MEM_Frame 0x0080 /* Value is a VdbeFrame object */
#define MEM_Undefined 0x0100 /* Value is undefined */
#define MEM_Cleared 0x0200 /* NULL set by OP_Null, not from data */
@@ -480,7 +479,7 @@ int sqlite3VdbeMemStringify(Mem *, u8);
int sqlite3VdbeIntValue(Mem *, int64_t *);
int sqlite3VdbeMemIntegerify(Mem *, bool is_forced);
int sqlite3VdbeRealValue(Mem *, double *);
-int sqlite3VdbeIntegerAffinity(Mem *);
+int mem_apply_integer_type(Mem *);
int sqlite3VdbeMemRealify(Mem *);
int sqlite3VdbeMemNumerify(Mem *);
int sqlite3VdbeMemCast(Mem *, enum field_type type);
diff --git a/src/box/sql/vdbeapi.c b/src/box/sql/vdbeapi.c
index 9e57af051..07c496207 100644
--- a/src/box/sql/vdbeapi.c
+++ b/src/box/sql/vdbeapi.c
@@ -280,7 +280,7 @@ sqlite3_value_type(sqlite3_value * pVal)
SQLITE_INTEGER, /* 0x1e */
SQLITE_NULL, /* 0x1f */
};
- return aType[pVal->flags & MEM_AffMask];
+ return aType[pVal->flags & MEM_TypeMask];
}
/* Make a copy of an sqlite3_value object
diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index b37a7141f..5f2cb8448 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -3507,7 +3507,7 @@ sqlite3VdbeGetBoundValue(Vdbe * v, int iVar, u8 aff)
sqlite3_value *pRet = sqlite3ValueNew(v->db);
if (pRet) {
sqlite3VdbeMemCopy((Mem *) pRet, pMem);
- sqlite3ValueApplyAffinity(pRet, aff);
+ sql_value_apply_type(pRet, aff);
}
return pRet;
}
diff --git a/src/box/sql/vdbemem.c b/src/box/sql/vdbemem.c
index a5dd7400a..2865fd68c 100644
--- a/src/box/sql/vdbemem.c
+++ b/src/box/sql/vdbemem.c
@@ -504,7 +504,7 @@ sqlite3VdbeRealValue(Mem * pMem, double *v)
* MEM_Int if we can.
*/
int
-sqlite3VdbeIntegerAffinity(Mem * pMem)
+mem_apply_integer_type(Mem *pMem)
{
int rc;
i64 ix;
@@ -584,7 +584,7 @@ sqlite3VdbeMemNumerify(Mem * pMem)
return SQLITE_ERROR;
pMem->u.r = v;
MemSetTypeFlag(pMem, MEM_Real);
- sqlite3VdbeIntegerAffinity(pMem);
+ mem_apply_integer_type(pMem);
}
}
assert((pMem->flags & (MEM_Int | MEM_Real | MEM_Null)) != 0);
@@ -593,10 +593,10 @@ sqlite3VdbeMemNumerify(Mem * pMem)
}
/*
- * Cast the datatype of the value in pMem according to the affinity
- * "aff". Casting is different from applying affinity in that a cast
+ * Cast the datatype of the value in pMem according to the type
+ * @type. Casting is different from applying type in that a cast
* is forced. In other words, the value is converted into the desired
- * affinity even if that results in loss of data. This routine is
+ * type even if that results in loss of data. This routine is
* used (for example) to implement the SQL "cast()" operator.
*/
int
@@ -643,7 +643,7 @@ sqlite3VdbeMemCast(Mem * pMem, enum field_type type)
assert(type == FIELD_TYPE_STRING);
assert(MEM_Str == (MEM_Blob >> 3));
pMem->flags |= (pMem->flags & MEM_Blob) >> 3;
- sqlite3ValueApplyAffinity(pMem, FIELD_TYPE_STRING);
+ sql_value_apply_type(pMem, FIELD_TYPE_STRING);
assert(pMem->flags & MEM_Str || pMem->db->mallocFailed);
pMem->flags &= ~(MEM_Int | MEM_Real | MEM_Blob | MEM_Zero);
return SQLITE_OK;
@@ -1152,7 +1152,7 @@ valueNew(sqlite3 * db, struct ValueNewStat4Ctx *p)
* error occurs, output parameter (*ppVal) is set to point to a value
* object containing the result before returning SQLITE_OK.
*
- * Affinity aff is applied to the result of the function before returning.
+ * Type @type is applied to the result of the function before returning.
* If the result is a text value, the sqlite3_value object uses encoding
* enc.
*
@@ -1222,7 +1222,7 @@ valueFromFunction(sqlite3 * db, /* The database connection */
rc = ctx.isError;
sqlite3ErrorMsg(pCtx->pParse, "%s", sqlite3_value_text(pVal));
} else {
- sqlite3ValueApplyAffinity(pVal, type);
+ sql_value_apply_type(pVal, type);
assert(rc == SQLITE_OK);
}
pCtx->pParse->rc = rc;
@@ -1285,7 +1285,7 @@ valueFromExpr(sqlite3 * db, /* The database connection */
testcase(rc != SQLITE_OK);
if (*ppVal) {
sqlite3VdbeMemCast(*ppVal, pExpr->type);
- sqlite3ValueApplyAffinity(*ppVal, type);
+ sql_value_apply_type(*ppVal, type);
}
return rc;
}
@@ -1318,9 +1318,9 @@ valueFromExpr(sqlite3 * db, /* The database connection */
}
if ((op == TK_INTEGER || op == TK_FLOAT) &&
type == FIELD_TYPE_SCALAR) {
- sqlite3ValueApplyAffinity(pVal, FIELD_TYPE_NUMBER);
+ sql_value_apply_type(pVal, FIELD_TYPE_NUMBER);
} else {
- sqlite3ValueApplyAffinity(pVal, type);
+ sql_value_apply_type(pVal, type);
}
if (pVal->flags & (MEM_Int | MEM_Real))
pVal->flags &= ~MEM_Str;
@@ -1339,7 +1339,7 @@ valueFromExpr(sqlite3 * db, /* The database connection */
} else {
pVal->u.i = -pVal->u.i;
}
- sqlite3ValueApplyAffinity(pVal, type);
+ sql_value_apply_type(pVal, type);
}
} else if (op == TK_NULL) {
pVal = valueNew(db, pCtx);
@@ -1502,7 +1502,7 @@ stat4ValueFromExpr(Parse * pParse, /* Parse context */
rc = sqlite3VdbeMemCopy((Mem *) pVal,
&v->aVar[iBindVar - 1]);
if (rc == SQLITE_OK) {
- sqlite3ValueApplyAffinity(pVal, type);
+ sql_value_apply_type(pVal, type);
}
pVal->db = pParse->db;
}
@@ -1536,7 +1536,7 @@ stat4ValueFromExpr(Parse * pParse, /* Parse context */
* vector components that match either of the two latter criteria listed
* above.
*
- * Before any value is appended to the record, the affinity of the
+ * Before any value is appended to the record, the type of the
* corresponding column within index pIdx is applied to it. Before
* this function returns, output parameter *pnExtract is set to the
* number of values appended to the record.
diff --git a/src/box/sql/where.c b/src/box/sql/where.c
index 6b3383bec..889a3517b 100644
--- a/src/box/sql/where.c
+++ b/src/box/sql/where.c
@@ -295,7 +295,7 @@ whereScanNext(WhereScan * pScan)
}
if ((pTerm->eOperator & pScan->
opMask) != 0) {
- /* Verify the affinity and collating sequence match */
+ /* Verify the type and collating sequence match */
if ((pTerm->eOperator & WO_ISNULL) == 0) {
pX = pTerm->pExpr;
if (!sql_expr_cmp_type_is_compatible(pX, pScan->idx_type))
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index ca375844f..7d657ad53 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -406,28 +406,25 @@ emit_apply_type(Parse *pParse, int base, int n, enum field_type *types)
}
}
-/*
- * Expression pRight, which is the RHS of a comparison operation, is
+/**
+ * Expression @rhs, which is the RHS of a comparison operation, is
* either a vector of n elements or, if n==1, a scalar expression.
- * Before the comparison operation, affinity zAff is to be applied
- * to the pRight values. This function modifies entries within the
+ * Before the comparison operation, types @types are to be applied
+ * to the @rhs values. This function modifies entries within the
* field sequence to SCALAR if either:
*
- * * the comparison will be performed with no affinity, or
- * * the affinity change in zAff is guaranteed not to change the value.
+ * * the comparison will be performed with no type, or
+ * * the type change in @types is guaranteed not to change the value.
*/
static void
-updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
- int n, /* Number of vector elements in comparison */
- enum field_type *zAff) /* Affinity string to modify */
+expr_cmp_update_rhs_type(struct Expr *rhs, int n, enum field_type *types)
{
- int i;
- for (i = 0; i < n; i++) {
- Expr *p = sqlite3VectorFieldSubexpr(pRight, i);
+ for (int i = 0; i < n; i++) {
+ Expr *p = sqlite3VectorFieldSubexpr(rhs, i);
enum field_type expr_type = sql_expr_type(p);
- if (sql_type_result(expr_type, zAff[i]) == FIELD_TYPE_SCALAR ||
- sql_expr_needs_no_type_change(p, zAff[i])) {
- zAff[i] = FIELD_TYPE_SCALAR;
+ if (sql_type_result(expr_type, types[i]) == FIELD_TYPE_SCALAR ||
+ sql_expr_needs_no_type_change(p, types[i])) {
+ types[i] = FIELD_TYPE_SCALAR;
}
}
}
@@ -651,7 +648,7 @@ codeEqualityTerm(Parse * pParse, /* The parsing context */
* In the example above nEq==2. But this subroutine works for any value
* of nEq including 0. If nEq==0, this routine is nearly a no-op.
* The only thing it does is allocate the pLevel->iMem memory cell and
- * compute the affinity string.
+ * compute the types array.
*
* The nExtraReg parameter is 0 or 1. It is 0 if all WHERE clause constraints
* are == or IN and are covered by the nEq. nExtraReg is 1 if there is
@@ -666,19 +663,19 @@ codeEqualityTerm(Parse * pParse, /* The parsing context */
* this routine allocates an additional nEq memory cells for internal
* use.
*
- * Before returning, *pzAff is set to point to a buffer containing a
- * copy of the column affinity string of the index allocated using
+ * Before returning, @types is set to point to a buffer containing a
+ * copy of the column types array of the index allocated using
* sqlite3DbMalloc(). Except, entries in the copy of the string associated
- * with equality constraints that use BLOB or NONE affinity are set to
+ * with equality constraints that use SCALAR type are set to
* SCALAR. This is to deal with SQL such as the following:
*
- * CREATE TABLE t1(a TEXT PRIMARY KEY, b);
+ * CREATE TABLE t1(a TEXT PRIMARY KEY, b BLOB);
* SELECT ... FROM t1 AS t2, t1 WHERE t1.a = t2.b;
*
- * In the example above, the index on t1(a) has TEXT affinity. But since
- * the right hand side of the equality constraint (t2.b) has BLOB/NONE affinity,
+ * In the example above, the index on t1(a) has STRING type. But since
+ * the right hand side of the equality constraint (t2.b) has SCALAR type,
* no conversion should be attempted before using a t2.b value as part of
- * a key to search the index. Hence the first byte in the returned affinity
+ * a key to search the index. Hence the first byte in the returned type
* string in this example would be set to SCALAR.
*/
static int
@@ -686,7 +683,7 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
WhereLevel * pLevel, /* Which nested loop of the FROM we are coding */
int bRev, /* Reverse the order of IN operators */
int nExtraReg, /* Number of extra registers to allocate */
- enum field_type **pzAff) /* OUT: Set to point to affinity string */
+ enum field_type **res_type)
{
u16 nEq; /* The number of == or IN constraints to code */
u16 nSkip; /* Number of left-most columns to skip */
@@ -710,8 +707,8 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
nReg = pLoop->nEq + nExtraReg;
pParse->nMem += nReg;
- enum field_type *zAff = sql_index_type_str(pParse->db, idx_def);
- assert(zAff != 0 || pParse->db->mallocFailed);
+ enum field_type *type = sql_index_type_str(pParse->db, idx_def);
+ assert(type != NULL || pParse->db->mallocFailed);
if (nSkip) {
int iIdxCur = pLevel->iIdxCur;
@@ -757,13 +754,13 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
}
if (pTerm->eOperator & WO_IN) {
if (pTerm->pExpr->flags & EP_xIsSelect) {
- /* No affinity ever needs to be (or should be) applied to a value
+ /* No type ever needs to be (or should be) applied to a value
* from the RHS of an "? IN (SELECT ...)" expression. The
* sqlite3FindInIndex() routine has already ensured that the
- * affinity of the comparison has been applied to the value.
+ * type of the comparison has been applied to the value.
*/
- if (zAff)
- zAff[j] = FIELD_TYPE_SCALAR;
+ if (type != NULL)
+ type[j] = FIELD_TYPE_SCALAR;
}
} else if ((pTerm->eOperator & WO_ISNULL) == 0) {
Expr *pRight = pTerm->pExpr->pRight;
@@ -772,20 +769,19 @@ codeAllEqualityTerms(Parse * pParse, /* Parsing context */
pLevel->addrBrk);
VdbeCoverage(v);
}
- if (zAff) {
- enum field_type type =
+ if (type != NULL) {
+ enum field_type rhs_type =
sql_expr_type(pRight);
- enum field_type idx_type = zAff[j];
- if (sql_type_result(type, idx_type) ==
+ if (sql_type_result(rhs_type, type[j]) ==
FIELD_TYPE_SCALAR) {
- zAff[j] = FIELD_TYPE_SCALAR;
+ type[j] = FIELD_TYPE_SCALAR;
}
- if (sql_expr_needs_no_type_change(pRight, idx_type))
- zAff[j] = FIELD_TYPE_SCALAR;
+ if (sql_expr_needs_no_type_change(pRight, type[j]))
+ type[j] = FIELD_TYPE_SCALAR;
}
}
}
- *pzAff = zAff;
+ *res_type = type;
return regBase;
}
@@ -994,8 +990,10 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
int iIdxCur; /* The VDBE cursor for the index */
int nExtraReg = 0; /* Number of extra registers needed */
int op; /* Instruction opcode */
- enum field_type *zStartAff; /* Affinity for start of range constraint */
- enum field_type *zEndAff = NULL; /* Affinity for end of range constraint */
+ /* Types for start of range constraint. */
+ enum field_type *start_types;
+ /* Types for end of range constraint */
+ enum field_type *end_types = NULL;
u8 bSeekPastNull = 0; /* True to seek past initial nulls */
u8 bStopAtNull = 0; /* Add condition to terminate at NULLs */
int force_integer_reg = -1; /* If non-negative: number of
@@ -1109,14 +1107,14 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
*/
regBase =
codeAllEqualityTerms(pParse, pLevel, bRev, nExtraReg,
- &zStartAff);
- if (zStartAff != NULL && nTop) {
+ &start_types);
+ if (start_types != NULL && nTop) {
uint32_t len = 0;
- for (enum field_type *tmp = &zStartAff[nEq];
+ for (enum field_type *tmp = &start_types[nEq];
*tmp != field_type_MAX; tmp++, len++);
uint32_t sz = len * sizeof(enum field_type);
- zEndAff = sqlite3DbMallocRaw(db, sz);
- memcpy(zEndAff, &zStartAff[nEq], sz);
+ end_types = sqlite3DbMallocRaw(db, sz);
+ memcpy(end_types, &start_types[nEq], sz);
}
addrNxt = pLevel->addrNxt;
@@ -1144,9 +1142,9 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
VdbeCoverage(v);
}
- if (zStartAff) {
- updateRangeAffinityStr(pRight, nBtm,
- &zStartAff[nEq]);
+ if (start_types) {
+ expr_cmp_update_rhs_type(pRight, nBtm,
+ &start_types[nEq]);
}
nConstraint += nBtm;
testcase(pRangeStart->wtFlags & TERM_VIRTUAL);
@@ -1191,7 +1189,7 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
}
}
emit_apply_type(pParse, regBase, nConstraint - bSeekPastNull,
- zStartAff);
+ start_types);
if (pLoop->nSkip > 0 && nConstraint == pLoop->nSkip) {
/* The skip-scan logic inside the call to codeAllEqualityConstraints()
* above has already left the cursor sitting on the correct row,
@@ -1241,10 +1239,10 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
addrNxt);
VdbeCoverage(v);
}
- if (zEndAff) {
- updateRangeAffinityStr(pRight, nTop, zEndAff);
+ if (end_types) {
+ expr_cmp_update_rhs_type(pRight, nTop, end_types);
emit_apply_type(pParse, regBase + nEq, nTop,
- zEndAff);
+ end_types);
} else {
assert(pParse->db->mallocFailed);
}
@@ -1261,8 +1259,8 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
endEq = 0;
nConstraint++;
}
- sqlite3DbFree(db, zStartAff);
- sqlite3DbFree(db, zEndAff);
+ sqlite3DbFree(db, start_types);
+ sqlite3DbFree(db, end_types);
/* Top of the loop body */
pLevel->p2 = sqlite3VdbeCurrentAddr(v);
diff --git a/src/box/sql/whereexpr.c b/src/box/sql/whereexpr.c
index a38ffed48..b016d18ef 100644
--- a/src/box/sql/whereexpr.c
+++ b/src/box/sql/whereexpr.c
@@ -272,7 +272,7 @@ like_optimization_is_valid(Parse *pParse, Expr *pExpr, Expr **ppPrefix,
sql_expr_type(pLeft) != FIELD_TYPE_STRING) {
/* IMP: R-02065-49465 The left-hand side of the
* LIKE operator must be the name of an indexed
- * column with TEXT affinity.
+ * column with STRING type.
*/
return 0;
}
@@ -749,7 +749,7 @@ exprAnalyzeOrTerm(SrcList * pSrc, /* the FROM clause */
} else if (pOrTerm->u.leftColumn != iColumn) {
okToChngToIN = 0;
} else {
- /* If the right-hand side is also a column, then the affinities
+ /* If the right-hand side is also a column, then the types
* of both right and left sides must be such that no type
* conversions are required on the right. (Ticket #2249)
*/
@@ -825,7 +825,7 @@ exprAnalyzeOrTerm(SrcList * pSrc, /* the FROM clause */
* 1. The SQLITE_Transitive optimization must be enabled
* 2. Must be either an == or an IS operator
* 3. Not originating in the ON clause of an OUTER JOIN
- * 4. The affinities of A and B must be compatible
+ * 4. The types of A and B must be compatible
* 5a. Both operands use the same collating sequence OR
* 5b. The overall collating sequence is BINARY
* If this routine returns TRUE, that means that the RHS can be substituted
>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
>> index d3a8644ce..b0a595b97 100644
>> --- a/src/box/sql/expr.c
>> +++ b/src/box/sql/expr.c
>> @@ -3213,7 +3226,9 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
>> if (rLhs != rLhsOrig)
>> sqlite3ReleaseTempReg(pParse, rLhs);
>> sqlite3ExprCachePop(pParse);
>> + sqlite3DbFree(pParse->db, aiMap);
>> VdbeComment((v, "end IN expr"));
>> + return;
>
> 1. What was wrong with the previous way of the function finalization?
> I do not see any functional changes in this function, but logic of
> freeing is different. Why? Because zAff should not be freed? Then please,
> fix it in the previous commit. And to do not change logic of
> "sqlite3ExprCodeIN_finished:" you can nullify zAff after passing it to
> OP_ApplyType above.
@@ -3133,9 +3147,8 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
* of the RHS using the LHS as a probe. If found, the result is
* true.
*/
- enum field_type *types = sql_affinity_str_to_field_type_str(zAff);
- types[nVector] = field_type_MAX;
- sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char *)types,
+ zAff[nVector] = field_type_MAX;
+ sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char*)zAff,
Before this patch, string (of types) passed to OP_ApplyType was allocated by
sql_affinity_str_to_field_type_str(), so it was different from zAff.
Now we pass directly zAff to OP_ApplyType, so we don’t need to free it.
On the other hand, indeed we can simply nullify it:
diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c
index fbada57b0..a9640a3a1 100644
--- a/src/box/sql/expr.c
+++ b/src/box/sql/expr.c
@@ -3150,6 +3150,11 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
zAff[nVector] = field_type_MAX;
sqlite3VdbeAddOp4(v, OP_ApplyType, rLhs, nVector, 0, (char*)zAff,
P4_DYNAMIC);
+ /*
+ * zAff will be freed at the end of VDBE execution, since
+ * it was passed with P4_DYNAMIC flag.
+ */
+ zAff = NULL;
if (destIfFalse == destIfNull) {
/* Combine Step 3 and Step 5 into a single opcode */
sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable,
@@ -3230,9 +3235,7 @@ sqlite3ExprCodeIN(Parse * pParse, /* Parsing and code generating context */
if (rLhs != rLhsOrig)
sqlite3ReleaseTempReg(pParse, rLhs);
sqlite3ExprCachePop(pParse);
- sqlite3DbFree(pParse->db, aiMap);
VdbeComment((v, "end IN expr"));
- return;
>> sqlite3ExprCodeIN_oom_error:
>> sqlite3DbFree(pParse->db, aiMap);
>> sqlite3DbFree(pParse->db, zAff);
>> diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
>> index 209e7bf0f..822cc40df 100644
>> --- a/src/box/sql/sqliteInt.h
>> +++ b/src/box/sql/sqliteInt.h
>> @@ -1807,7 +1801,7 @@ struct Savepoint {
>> * operator is NULL. It is added to certain comparison operators to
>> * prove that the operands are always NOT NULL.
>> */
>> -#define SQLITE_KEEPNULL 0x08 /* Used by vector == or <> */
>> +#define SQLITE_KEEPNULL 0x40 /* Used by vector == or <> */
>> #define SQLITE_JUMPIFNULL 0x10 /* jumps if either operand is NULL */
>> #define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
>> #define SQLITE_NULLEQ 0x80 /* NULL=NULL */
>
> 2. Please, keep them sorted. Cuts the eye.
diff --git a/src/box/sql/sqliteInt.h b/src/box/sql/sqliteInt.h
index 850eea893..e3b5cac67 100644
--- a/src/box/sql/sqliteInt.h
+++ b/src/box/sql/sqliteInt.h
@@ -1801,9 +1801,9 @@ struct Savepoint {
* operator is NULL. It is added to certain comparison operators to
* prove that the operands are always NOT NULL.
*/
-#define SQLITE_KEEPNULL 0x40 /* Used by vector == or <> */
#define SQLITE_JUMPIFNULL 0x10 /* jumps if either operand is NULL */
#define SQLITE_STOREP2 0x20 /* Store result in reg[P2] rather than jump */
+#define SQLITE_KEEPNULL 0x40 /* Used by vector == or <> */
#define SQLITE_NULLEQ 0x80 /* NULL=NULL */
#define SQLITE_NOTNULL 0x90 /* Assert that operands are never NULL */
>> @@ -2615,7 +2609,8 @@ struct Select {
>> */
>> struct SelectDest {
>> u8 eDest; /* How to dispose of the results. On of SRT_* above. */
>> - char *zAffSdst; /* Affinity used when eDest==SRT_Set */
>> + /* Affinity used when eDest==SRT_Set */
>> + enum field_type *zAffSdst;
>
> 3. Not affinity.
@@ -2609,7 +2609,7 @@ struct Select {
*/
struct SelectDest {
u8 eDest; /* How to dispose of the results. On of SRT_* above. */
- /* Affinity used when eDest==SRT_Set */
- enum field_type *zAffSdst;
+ /** Type used when eDest==SRT_Set */
+ enum field_type *dest_type;
int iSDParm; /* A parameter used by the eDest disposal method */
/** Register containing ephemeral's space pointer. */
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 61d73b676..c3f596d4f 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -2163,9 +2151,16 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
>> break;
>> }
>> } else {
>> - /* Neither operand is NULL. Do a comparison. */
>> - affinity = pOp->p5 & AFFINITY_MASK;
>> - if (affinity>=AFFINITY_INTEGER) {
>> + /*
>> + * Neither operand is NULL. Do a comparison.
>> + * 15 is 1111 in a binary form.
>> + * Since all existing types can be fitted in 4 bits
>> + * (field_type_MAX == 10), it is enough to 'and'
>> + * p5 with this constant. Node that all other flags
>> + * that can be stored in p5 are >= 16.
>> + */
>> + affinity = pOp->p5 & 15;
>
> 4. Honestly, I did not expect it from you. Please, never use such
> constants and assumptions about them in code. field_type_MAX = 10 can
> be changed any day (for example, when we introduce decimal as a native
> Tarantool data type, or decide to add datetime). In fact, it is already
> not 10 - it is 9.
Oh, cmon. This place will explode only if number of types exceed 15.
Not so soon, I guess (when was the last time we introduced new type?).
So it won’t be my problem by the time it occurs.
> I propose you to either come up with a better solution or to use one of
> my solutions:
>
> * return AFFINITY_MASK as P5_FIELD_TYPE_MASK, and add a static assertion
> that P5_FIELD_TYPE_MASK < (1 << bit_count(field_type)).
>
> Or write in field_def.h something like this:
>
> static_assert(bit_count(field_type) <= 4,
> "size of enum field_type is used in "\
> "VdbeOp.p5 to fit it into 4 bits”);
To be honest I don’t understand these bit assertions.
Why can’t we simply check that field_type_MAX <= 15?
diff --git a/src/box/field_def.c b/src/box/field_def.c
index b34d2ccd9..1a8074650 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -33,6 +33,13 @@
#include "trivia/util.h"
#include "key_def.h"
+/**
+ * For detailed explanation see context of OP_Eq, OP_Lt etc
+ * opcodes in vdbe.c.
+ */
+static_assert(field_type_MAX <= 15, "values of enum field_type "\
+ "should fit into 4 bits of VdbeOp.p5");
+
Also, as you may notice, I placed it to field_def.c
Seems that field_def.h is used to generate module.h,
which in turn doesn’t tolerate this kind of assert.
> To get bit_count during compilation you can specify it explicitly right
> after field_type_MAX in enum field_type so as to change it always
> together with field_type_MAX.
>
>> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
>> index 33b860f36..01c1a2c6c 100644
>> --- a/src/box/sql/wherecode.c
>> +++ b/src/box/sql/wherecode.c
>> @@ -367,15 +367,15 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
>> * Code an OP_ApplyType opcode to apply the column type string types
>> * to the n registers starting at base.
>> *
>> - * As an optimization, AFFINITY_BLOB entries (which are no-ops) at the
>> + * As an optimization, SCALAR entries (which are no-ops) at the
>> * beginning and end of zAff are ignored. If all entries in zAff are
>> - * AFFINITY_BLOB, then no code gets generated.
>> + * SCALAR, then no code gets generated.
>> *
>> * This routine makes its own copy of zAff so that the caller is free
>> * to modify zAff after this routine returns.
>> */
>> static void
>> -codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
>> +codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
>
> 5. Not affinity.
Ok, refactored whole function:
diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
index 53445faf5..1f28893f6 100644
--- a/src/box/sql/wherecode.c
+++ b/src/box/sql/wherecode.c
@@ -363,22 +363,22 @@ disableTerm(WhereLevel * pLevel, WhereTerm * pTerm)
}
}
-/*
- * Code an OP_ApplyType opcode to apply the column type string types
- * to the n registers starting at base.
+/**
+ * Code an OP_ApplyType opcode to apply the column type string
+ * @types to the n registers starting at @base.
*
* As an optimization, SCALAR entries (which are no-ops) at the
- * beginning and end of zAff are ignored. If all entries in zAff are
- * SCALAR, then no code gets generated.
+ * beginning and end of @types are ignored. If all entries in
+ * @types are SCALAR, then no code gets generated.
*
- * This routine makes its own copy of zAff so that the caller is free
- * to modify zAff after this routine returns.
+ * This routine makes its own copy of @types so that the caller is
+ * free to modify @types after this routine returns.
*/
static void
-codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
+emit_apply_type(Parse *pParse, int base, int n, enum field_type *types)
{
Vdbe *v = pParse->pVdbe;
- if (zAff == 0) {
+ if (types == NULL) {
assert(pParse->db->mallocFailed);
return;
}
@@ -388,17 +388,17 @@ codeApplyAffinity(Parse * pParse, int base, int n, enum field_type *zAff)
* Adjust base and n to skip over SCALAR entries at the
* beginning and end of the type sequence.
*/
- while (n > 0 && zAff[0] == FIELD_TYPE_SCALAR) {
+ while (n > 0 && types[0] == FIELD_TYPE_SCALAR) {
n--;
base++;
- zAff++;
+ types++;
}
- while (n > 1 && zAff[n - 1] == FIELD_TYPE_SCALAR) {
+ while (n > 1 && types[n - 1] == FIELD_TYPE_SCALAR) {
n--;
}
if (n > 0) {
- enum field_type *types = field_type_sequence_dup(pParse, zAff,
- n);
- sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0, (char *) types,
- P4_DYNAMIC);
+ enum field_type *types_dup = field_type_sequence_dup(pParse,
+ types, n);
+ sqlite3VdbeAddOp4(v, OP_ApplyType, base, n, 0,
+ (char *) types_dup, P4_DYNAMIC);
@@ -1190,8 +1190,8 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
}
}
}
- codeApplyAffinity(pParse, regBase, nConstraint - bSeekPastNull,
- zStartAff);
+ emit_apply_type(pParse, regBase, nConstraint - bSeekPastNull,
+ zStartAff);
if (pLoop->nSkip > 0 && nConstraint == pLoop->nSkip) {
/* The skip-scan logic inside the call to codeAllEqualityConstraints()
* above has already left the cursor sitting on the correct row,
@@ -1243,8 +1243,8 @@ sqlite3WhereCodeOneLoopStart(WhereInfo * pWInfo, /* Complete information about t
}
if (zEndAff) {
updateRangeAffinityStr(pRight, nTop, zEndAff);
- codeApplyAffinity(pParse, regBase + nEq, nTop,
- zEndAff);
+ emit_apply_type(pParse, regBase + nEq, nTop,
+ zEndAff);
} else {
assert(pParse->db->mallocFailed);
}
>> @@ -419,16 +419,15 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
>> static void
>> updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
>> int n, /* Number of vector elements in comparison */
>> - char *zAff) /* Affinity string to modify */
>> + enum field_type *zAff) /* Affinity string to modify */
>
> 6. The same. In many other places too. Where you changed type and
> usages of a variable, but kept its name.
I did it on purpose - to reduce diff, so that separate functional
changes and non-functional such as renaming. Now fixed.
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2019-01-28 16:40 ` n.pettik
@ 2019-01-30 13:04 ` Vladislav Shpilevoy
2019-02-01 16:39 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-01-30 13:04 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes! See comments below.
>>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>>> index 61d73b676..c3f596d4f 100644
>>> --- a/src/box/sql/vdbe.c
>>> +++ b/src/box/sql/vdbe.c
>>> @@ -2163,9 +2151,16 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
>>> break;
>>> }
>>> } else {
>>> - /* Neither operand is NULL. Do a comparison. */
>>> - affinity = pOp->p5 & AFFINITY_MASK;
>>> - if (affinity>=AFFINITY_INTEGER) {
>>> + /*
>>> + * Neither operand is NULL. Do a comparison.
>>> + * 15 is 1111 in a binary form.
>>> + * Since all existing types can be fitted in 4 bits
>>> + * (field_type_MAX == 10), it is enough to 'and'
>>> + * p5 with this constant. Node that all other flags
>>> + * that can be stored in p5 are >= 16.
>>> + */
>>> + affinity = pOp->p5 & 15;
>>
>> 4. Honestly, I did not expect it from you. Please, never use such
>> constants and assumptions about them in code. field_type_MAX = 10 can
>> be changed any day (for example, when we introduce decimal as a native
>> Tarantool data type, or decide to add datetime). In fact, it is already
>> not 10 - it is 9.
>
> Oh, cmon. This place will explode only if number of types exceed 15.
> Not so soon, I guess (when was the last time we introduced new type?).
> So it won’t be my problem by the time it occurs.
>
>> I propose you to either come up with a better solution or to use one of
>> my solutions:
>>
>> * return AFFINITY_MASK as P5_FIELD_TYPE_MASK, and add a static assertion
>> that P5_FIELD_TYPE_MASK < (1 << bit_count(field_type)).
>>
>> Or write in field_def.h something like this:
>>
>> static_assert(bit_count(field_type) <= 4,
>> "size of enum field_type is used in "\
>> "VdbeOp.p5 to fit it into 4 bits”);
>
> To be honest I don’t understand these bit assertions.
> Why can’t we simply check that field_type_MAX <= 15?
>
> diff --git a/src/box/field_def.c b/src/box/field_def.c
> index b34d2ccd9..1a8074650 100644
> --- a/src/box/field_def.c
> +++ b/src/box/field_def.c
> @@ -33,6 +33,13 @@
> #include "trivia/util.h"
> #include "key_def.h"
>
> +/**
> + * For detailed explanation see context of OP_Eq, OP_Lt etc
> + * opcodes in vdbe.c.
> + */
> +static_assert(field_type_MAX <= 15, "values of enum field_type "\
> + "should fit into 4 bits of VdbeOp.p5");
> +
>
> Also, as you may notice, I placed it to field_def.c
> Seems that field_def.h is used to generate module.h,
> which in turn doesn’t tolerate this kind of assert.
field_def.h is not copied to module.h entirely. Only
the code between /** \cond public */ and /** \endcond public */.
Please, put this assertion below /** \endcond public */.
>
>> To get bit_count during compilation you can specify it explicitly right
>> after field_type_MAX in enum field_type so as to change it always
>> together with field_type_MAX.
>>
> diff --git a/src/box/sql/wherecode.c b/src/box/sql/wherecode.c
> index 53445faf5..1f28893f6 100644
> --- a/src/box/sql/wherecode.c
> +++ b/src/box/sql/wherecode.c
>>> @@ -419,16 +419,15 @@ codeApplyAffinity(Parse * pParse, int base, int n, char *zAff)
>>> static void
>>> updateRangeAffinityStr(Expr * pRight, /* RHS of comparison */
>>> int n, /* Number of vector elements in comparison */
>>> - char *zAff) /* Affinity string to modify */
>>> + enum field_type *zAff) /* Affinity string to modify */
>>
>> 6. The same. In many other places too. Where you changed type and
>> usages of a variable, but kept its name.
>
> I did it on purpose - to reduce diff, so that separate functional
> changes and non-functional such as renaming. Now fixed.
Small diff is good, but lets do not overemphasize. This patch states
that affinity is removed, and not only as a functionality, but from
the source code, but before your fixes it actually was not. It
complicated reading and understanding of the code.
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 80d2fd0aa..358b69754 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -2164,17 +2151,24 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
> break;
> }
> } else {
> - /* Neither operand is NULL. Do a comparison. */
> - affinity = pOp->p5 & AFFINITY_MASK;
> - if (affinity>=AFFINITY_INTEGER) {
> + /*
> + * Neither operand is NULL. Do a comparison.
> + * 15 is 1111 in a binary form.
> + * Since all existing types can be fitted in 4 bits
> + * (field_type_MAX == 10), it is enough to 'and'
> + * p5 with this constant. Node that all other flags
> + * that can be stored in p5 are >= 16.
Firstly, as I said in the previous review, field_type_MAX == 9.
Secondly, please, do not use a constant. All of them (15, 111, 10, 16).
Instead of this huge comment full of constants just create a enum
value with an appropriate name, like FIELD_TYPE_MASK = 15, it is
even ok to put this name into field_def.h. Thirdly, not all p5 flags
are >= 16. But for these concrete opcodes it is so, luckily. At
least we have OPFLAG_NCHANGE, OPFLAG_EPHEM, OPFLAG_SEEKEQ etc -
they all are <= 15.
> + */
> + enum field_type type = pOp->p5 & 15;
> + if (sql_type_is_numeric(type)) {
> if ((flags1 | flags3)&MEM_Str) {
> if ((flags1 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
> - applyNumericAffinity(pIn1,0);
> + mem_apply_numeric_type(pIn1, 0);
> testcase( flags3!=pIn3->flags); /* Possible if pIn1==pIn3 */
> flags3 = pIn3->flags;
> }
> if ((flags3 & (MEM_Int|MEM_Real|MEM_Str))==MEM_Str) {
> - if (applyNumericAffinity(pIn3,0) != 0) {
> + if (mem_apply_numeric_type(pIn3, 0) != 0) {
> diag_set(ClientError,
> ER_SQL_TYPE_MISMATCH,
> sqlite3_value_text(pIn3),
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2019-01-30 13:04 ` Vladislav Shpilevoy
@ 2019-02-01 16:39 ` n.pettik
2019-02-05 15:08 ` Vladislav Shpilevoy
0 siblings, 1 reply; 48+ messages in thread
From: n.pettik @ 2019-02-01 16:39 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>>> I propose you to either come up with a better solution or to use one of
>>> my solutions:
>>>
>>> * return AFFINITY_MASK as P5_FIELD_TYPE_MASK, and add a static assertion
>>> that P5_FIELD_TYPE_MASK < (1 << bit_count(field_type)).
>>>
>>> Or write in field_def.h something like this:
>>>
>>> static_assert(bit_count(field_type) <= 4,
>>> "size of enum field_type is used in "\
>>> "VdbeOp.p5 to fit it into 4 bits”);
>> To be honest I don’t understand these bit assertions.
>> Why can’t we simply check that field_type_MAX <= 15?
>> diff --git a/src/box/field_def.c b/src/box/field_def.c
>> index b34d2ccd9..1a8074650 100644
>> --- a/src/box/field_def.c
>> +++ b/src/box/field_def.c
>> @@ -33,6 +33,13 @@
>> #include "trivia/util.h"
>> #include "key_def.h"
>> +/**
>> + * For detailed explanation see context of OP_Eq, OP_Lt etc
>> + * opcodes in vdbe.c.
>> + */
>> +static_assert(field_type_MAX <= 15, "values of enum field_type "\
>> + "should fit into 4 bits of VdbeOp.p5");
>> +
>> Also, as you may notice, I placed it to field_def.c
>> Seems that field_def.h is used to generate module.h,
>> which in turn doesn’t tolerate this kind of assert.
>
> field_def.h is not copied to module.h entirely. Only
> the code between /** \cond public */ and /** \endcond public */.
> Please, put this assertion below /** \endcond public */.
Didn’t pay attention to that. Fixed:
diff --git a/src/box/field_def.c b/src/box/field_def.c
index 1a8074650..b34d2ccd9 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -33,13 +33,6 @@
#include "trivia/util.h"
#include "key_def.h"
-/**
- * For detailed explanation see context of OP_Eq, OP_Lt etc
- * opcodes in vdbe.c.
- */
-static_assert(field_type_MAX <= 15, "values of enum field_type "\
- "should fit into 4 bits of VdbeOp.p5");
-
const char *mp_type_strs[] = {
/* .MP_NIL = */ "nil",
/* .MP_UINT = */ "unsigned",
diff --git a/src/box/field_def.h b/src/box/field_def.h
index 93e38ea55..358e35f69 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -87,6 +87,13 @@ affinity_type_str(enum affinity_type type);
/** \endcond public */
+/**
+ * For detailed explanation see context of OP_Eq, OP_Lt etc
+ * opcodes in vdbe.c.
+ */
+static_assert(field_type_MAX <= 15, "values of enum field_type "\
+ "should fit into 4 bits of VdbeOp.p5");
+
extern const char *field_type_strs[];
extern const char *on_conflict_action_strs[];
>> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
>> index 80d2fd0aa..358b69754 100644
>> --- a/src/box/sql/vdbe.c
>> +++ b/src/box/sql/vdbe.c
>> @@ -2164,17 +2151,24 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
>> break;
>> }
>> } else {
>> - /* Neither operand is NULL. Do a comparison. */
>> - affinity = pOp->p5 & AFFINITY_MASK;
>> - if (affinity>=AFFINITY_INTEGER) {
>> + /*
>> + * Neither operand is NULL. Do a comparison.
>> + * 15 is 1111 in a binary form.
>> + * Since all existing types can be fitted in 4 bits
>> + * (field_type_MAX == 10), it is enough to 'and'
>> + * p5 with this constant. Node that all other flags
>> + * that can be stored in p5 are >= 16.
>
> Firstly, as I said in the previous review, field_type_MAX == 9.
> Secondly, please, do not use a constant. All of them (15, 111, 10, 16).
> Instead of this huge comment full of constants just create a enum
> value with an appropriate name, like FIELD_TYPE_MASK = 15, it is
> even ok to put this name into field_def.h. Thirdly, not all p5 flags
> are >= 16. But for these concrete opcodes it is so, luckily. At
> least we have OPFLAG_NCHANGE, OPFLAG_EPHEM, OPFLAG_SEEKEQ etc -
> they all are <= 15.
Is this variant OK?
+/**
+ * This mask allows to store in p5 operand of OP_Eq, OP_Lt etc
+ * opcodes field type alonside with other flags.
+ */
+enum field_type_mask {
+ FIELD_TYPE_MASK = 15
+};
+
extern const char *field_type_strs[];
extern const char *on_conflict_action_strs[];
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index 358b69754..dae05e80f 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2153,13 +2153,13 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
} else {
/*
* Neither operand is NULL. Do a comparison.
- * 15 is 1111 in a binary form.
- * Since all existing types can be fitted in 4 bits
- * (field_type_MAX == 10), it is enough to 'and'
- * p5 with this constant. Node that all other flags
- * that can be stored in p5 are >= 16.
+ * Since all existing types can be fitted in 4
+ * bits (field_type_MAX == 9), it is enough to
+ * 'and' p5 with field mask. Note that values of
+ * other flags that can be stored in p5 of these
+ * opcodes are >= FIELD_TYPE_MASK.
*/
- enum field_type type = pOp->p5 & 15;
+ enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2019-02-01 16:39 ` n.pettik
@ 2019-02-05 15:08 ` Vladislav Shpilevoy
2019-02-05 17:46 ` n.pettik
0 siblings, 1 reply; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-02-05 15:08 UTC (permalink / raw)
To: n.pettik, tarantool-patches
Thanks for the fixes!
> diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
> index 358b69754..dae05e80f 100644
> --- a/src/box/sql/vdbe.c
> +++ b/src/box/sql/vdbe.c
> @@ -2153,13 +2153,13 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
> } else {
> /*
> * Neither operand is NULL. Do a comparison.
'Neither operand is NULL' - this is 'else' branch of
"(flags1 | flags3)&MEM_Null", so I guess it is obvious.
> - * 15 is 1111 in a binary form.
> - * Since all existing types can be fitted in 4 bits
> - * (field_type_MAX == 10), it is enough to 'and'
> - * p5 with this constant. Node that all other flags
> - * that can be stored in p5 are >= 16.
> + * Since all existing types can be fitted in 4
> + * bits (field_type_MAX == 9), it is enough to
> + * 'and' p5 with field mask. Note that values of
> + * other flags that can be stored in p5 of these
> + * opcodes are >= FIELD_TYPE_MASK.
They can not be == FIELD_TYPE_MASK. Please, again, do not try to
write here any constants, including bit counts. A definition named
MASK is already enough to make it clear, that p5 'and' mask give a
type.
> */
> - enum field_type type = pOp->p5 & 15;
> + enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
>
Please, apply the diff below. I made enum field_type_mask an
anonymous enum, as we usually do with independent constants. Also, I
used FIELD_TYPE_MASK instead of 15 in static assertion, and removed
the comment from vdbe.c about what a mask is.
diff --git a/src/box/field_def.h b/src/box/field_def.h
index 3b8eb0ccf..3eda0e38f 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -87,21 +87,22 @@ affinity_type_str(enum affinity_type type);
/** \endcond public */
+enum {
+ /**
+ * This mask allows to store in VdbeOp.p5 operand of
+ * OP_Eq, OP_Lt etc opcodes field type alonside with
+ * flags.
+ */
+ FIELD_TYPE_MASK = 15
+};
+
/**
* For detailed explanation see context of OP_Eq, OP_Lt etc
* opcodes in vdbe.c.
*/
-static_assert(field_type_MAX <= 15, "values of enum field_type "\
+static_assert(field_type_MAX <= FIELD_TYPE_MASK, "values of enum field_type "\
"should fit into 4 bits of VdbeOp.p5");
-/**
- * This mask allows to store in p5 operand of OP_Eq, OP_Lt etc
- * opcodes field type alonside with other flags.
- */
-enum field_type_mask {
- FIELD_TYPE_MASK = 15
-};
-
extern const char *field_type_strs[];
extern const char *on_conflict_action_strs[];
diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c
index dae05e80f..a66ab4992 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -2151,14 +2151,6 @@ case OP_Ge: { /* same as TK_GE, jump, in1, in3 */
break;
}
} else {
- /*
- * Neither operand is NULL. Do a comparison.
- * Since all existing types can be fitted in 4
- * bits (field_type_MAX == 9), it is enough to
- * 'and' p5 with field mask. Note that values of
- * other flags that can be stored in p5 of these
- * opcodes are >= FIELD_TYPE_MASK.
- */
enum field_type type = pOp->p5 & FIELD_TYPE_MASK;
if (sql_type_is_numeric(type)) {
if ((flags1 | flags3)&MEM_Str) {
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 7/8] sql: clean-up affinity from SQL source code
2019-02-05 15:08 ` Vladislav Shpilevoy
@ 2019-02-05 17:46 ` n.pettik
0 siblings, 0 replies; 48+ messages in thread
From: n.pettik @ 2019-02-05 17:46 UTC (permalink / raw)
To: tarantool-patches; +Cc: Vladislav Shpilevoy
>> - * 15 is 1111 in a binary form.
>> - * Since all existing types can be fitted in 4 bits
>> - * (field_type_MAX == 10), it is enough to 'and'
>> - * p5 with this constant. Node that all other flags
>> - * that can be stored in p5 are >= 16.
>> + * Since all existing types can be fitted in 4
>> + * bits (field_type_MAX == 9), it is enough to
>> + * 'and' p5 with field mask. Note that values of
>> + * other flags that can be stored in p5 of these
>> + * opcodes are >= FIELD_TYPE_MASK.
>
> They can not be == FIELD_TYPE_MASK. Please, again, do not try to
> write here any constants, including bit counts. A definition named
> MASK is already enough to make it clear, that p5 'and' mask give a
> type.
Ok, I won’t argue.
> Please, apply the diff below. I made enum field_type_mask an
> anonymous enum, as we usually do with independent constants. Also, I
> used FIELD_TYPE_MASK instead of 15 in static assertion, and removed
> the comment from vdbe.c about what a mask is.
Seems like compilers don’t allow to compare two different enums,
even though their values are defined:
/tarantool/src/box/field_def.h:91:30: error: comparison between ‘enum field_type’ and ‘enum <anonymous>’ [-Werror=enum-compare]
static_assert(field_type_MAX <= FIELD_TYPE_MASK, "values of enum field_type "\
So I added explicit cast:
diff --git a/src/box/field_def.h b/src/box/field_def.h
index dc7730f9f..f944de9d6 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -88,8 +88,8 @@ enum {
* For detailed explanation see context of OP_Eq, OP_Lt etc
* opcodes in vdbe.c.
*/
-static_assert(field_type_MAX <= FIELD_TYPE_MASK, "values of enum field_type "\
- "should fit into 4 bits of VdbeOp.p5");
+static_assert((int) field_type_MAX <= (int) FIELD_TYPE_MASK,
+ "values of enum field_type should fit into 4 bits of VdbeOp.p5”);
The rest of diff applied as is.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [tarantool-patches] [PATCH 8/8] Remove affinity from field definition
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (6 preceding siblings ...)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 7/8] sql: clean-up affinity from SQL source code Nikita Pettik
@ 2018-12-28 9:34 ` Nikita Pettik
2019-02-05 19:41 ` [tarantool-patches] Re: [PATCH 0/8] Eliminate affinity from source code Vladislav Shpilevoy
2019-02-08 13:37 ` Kirill Yukhin
9 siblings, 0 replies; 48+ messages in thread
From: Nikita Pettik @ 2018-12-28 9:34 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Closes #3698
---
src/box/field_def.c | 20 --------------------
src/box/field_def.h | 17 -----------------
2 files changed, 37 deletions(-)
diff --git a/src/box/field_def.c b/src/box/field_def.c
index b34d2ccd9..0ba3d3294 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -47,15 +47,6 @@ const char *mp_type_strs[] = {
/* .MP_EXT = */ "extension",
};
-static const char *affinity_type_strs[] = {
- /* [UNDEFINED] */ "undefined",
- /* [BLOB - 'A'] */ "blob",
- /* [TEXT - 'A'] */ "text",
- /* [NUMERIC - 'A'] */ "numeric",
- /* [INTEGER - 'A'] */ "integer",
- /* [REAL - 'A'] */ "real",
-};
-
const uint32_t field_mp_type[] = {
/* [FIELD_TYPE_ANY] = */ UINT32_MAX,
/* [FIELD_TYPE_UNSIGNED] = */ 1U << MP_UINT,
@@ -71,15 +62,6 @@ const uint32_t field_mp_type[] = {
/* [FIELD_TYPE_MAP] = */ (1U << MP_MAP),
};
-const char *
-affinity_type_str(enum affinity_type type)
-{
- if (type < 'A')
- return affinity_type_strs[type];
- else
- return affinity_type_strs[type - 'A' + 1];
-}
-
const char *field_type_strs[] = {
/* [FIELD_TYPE_ANY] = */ "any",
/* [FIELD_TYPE_UNSIGNED] = */ "unsigned",
@@ -142,13 +124,11 @@ const struct opt_def field_def_reg[] = {
nullable_action, NULL),
OPT_DEF("collation", OPT_UINT32, struct field_def, coll_id),
OPT_DEF("default", OPT_STRPTR, struct field_def, default_value),
- OPT_DEF("affinity", OPT_UINT32, struct field_def, affinity),
OPT_END,
};
const struct field_def field_def_default = {
.type = FIELD_TYPE_ANY,
- .affinity = AFFINITY_UNDEFINED,
.name = NULL,
.is_nullable = false,
.nullable_action = ON_CONFLICT_ACTION_DEFAULT,
diff --git a/src/box/field_def.h b/src/box/field_def.h
index 93e38ea55..9a3d0b1ac 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -73,18 +73,6 @@ enum on_conflict_action {
on_conflict_action_MAX
};
-enum affinity_type {
- AFFINITY_UNDEFINED = 0,
- AFFINITY_BLOB = 'A',
- AFFINITY_TEXT = 'B',
- AFFINITY_INTEGER = 'D',
- AFFINITY_REAL = 'E',
-};
-
-/** String name of @a type. */
-const char *
-affinity_type_str(enum affinity_type type);
-
/** \endcond public */
extern const char *field_type_strs[];
@@ -123,11 +111,6 @@ struct field_def {
* then UNKNOWN is stored for it.
*/
enum field_type type;
- /**
- * Affinity type for comparations in SQL.
- * FIXME: Remove affinity after types redesign in SQL.
- */
- enum affinity_type affinity;
/** 0-terminated field name. */
char *name;
/** True, if a field can store NULL. */
--
2.15.1
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 0/8] Eliminate affinity from source code
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (7 preceding siblings ...)
2018-12-28 9:34 ` [tarantool-patches] [PATCH 8/8] Remove affinity from field definition Nikita Pettik
@ 2019-02-05 19:41 ` Vladislav Shpilevoy
2019-02-08 13:37 ` Kirill Yukhin
9 siblings, 0 replies; 48+ messages in thread
From: Vladislav Shpilevoy @ 2019-02-05 19:41 UTC (permalink / raw)
To: tarantool-patches, Nikita Pettik
Thank you for all the diligent labour on this patchset.
Now LGTM.
On 28/12/2018 10:34, Nikita Pettik wrote:
> Branch: https://github.com/tarantool/tarantool/tree/np/gh-3698-eliminate-affinity
> Issue:
> https://github.com/tarantool/tarantool/issues/3698
> https://github.com/tarantool/tarantool/issues/3886
>
> Main purpose of current patch-set is to remove affinity from source
> code. We are starting from introducing itermediate converstors
> from affinity to field type and vice versa. Then, step-by-step we
> are replacing affinity usages with field type. Non-trivial part of
> this patch-set is removing numeric affinity (see third patch in
> series and commit message).
>
> It worth noting that current patch-set also fixes issue connected
> with invisible to query optimizer indexes, which are created from
> Lua: to choose suitable index QO checked affinity compatibility of
> key part and field def. As a result, indexes created without affinity
> field couldn't be used to process query.
>
> Nikita Pettik (8):
> sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define
> sql: use field type instead of affinity for type_def
> sql: remove numeric affinity
> sql: replace affinity with field type for func
> sql: replace field type with affinity for VDBE runtime
> sql: replace affinity with field type in struct Expr
> sql: clean-up affinity from SQL source code
> Remove affinity from field definition
>
> src/box/field_def.c | 20 ---
> src/box/field_def.h | 18 ---
> src/box/lua/lua_sql.c | 12 +-
> src/box/sql.c | 14 +-
> src/box/sql/analyze.c | 5 +-
> src/box/sql/build.c | 19 +--
> src/box/sql/date.c | 17 ++-
> src/box/sql/delete.c | 16 +-
> src/box/sql/expr.c | 281 ++++++++++++++++-------------------
> src/box/sql/fkey.c | 8 +-
> src/box/sql/func.c | 91 ++++++------
> src/box/sql/insert.c | 60 +++-----
> src/box/sql/main.c | 4 +-
> src/box/sql/parse.y | 38 +++--
> src/box/sql/resolve.c | 24 +--
> src/box/sql/select.c | 29 ++--
> src/box/sql/sqliteInt.h | 107 +++++--------
> src/box/sql/update.c | 17 +--
> src/box/sql/vdbe.c | 120 +++++++--------
> src/box/sql/vdbeInt.h | 2 +-
> src/box/sql/vdbeaux.c | 19 ++-
> src/box/sql/vdbemem.c | 37 ++---
> src/box/sql/where.c | 37 ++---
> src/box/sql/whereInt.h | 3 +-
> src/box/sql/wherecode.c | 59 +++-----
> src/box/sql/whereexpr.c | 34 ++---
> test/sql-tap/cast.test.lua | 10 +-
> test/sql-tap/lua-tables.test.lua | 38 ++++-
> test/sql-tap/tkt-80e031a00f.test.lua | 12 +-
> test/sql/iproto.result | 2 +-
> test/sql/types.result | 15 +-
> test/sql/upgrade.result | 6 +-
> 32 files changed, 516 insertions(+), 658 deletions(-)
>
> --
> 2.15.1
>
>
^ permalink raw reply [flat|nested] 48+ messages in thread* [tarantool-patches] Re: [PATCH 0/8] Eliminate affinity from source code
2018-12-28 9:34 [tarantool-patches] [PATCH 0/8] Eliminate affinity from source code Nikita Pettik
` (8 preceding siblings ...)
2019-02-05 19:41 ` [tarantool-patches] Re: [PATCH 0/8] Eliminate affinity from source code Vladislav Shpilevoy
@ 2019-02-08 13:37 ` Kirill Yukhin
9 siblings, 0 replies; 48+ messages in thread
From: Kirill Yukhin @ 2019-02-08 13:37 UTC (permalink / raw)
To: tarantool-patches; +Cc: v.shpilevoy, Nikita Pettik
Hello,
On 28 Dec 11:34, Nikita Pettik wrote:
> Branch: https://github.com/tarantool/tarantool/tree/np/gh-3698-eliminate-affinity
> Issue:
> https://github.com/tarantool/tarantool/issues/3698
> https://github.com/tarantool/tarantool/issues/3886
>
> Main purpose of current patch-set is to remove affinity from source
> code. We are starting from introducing itermediate converstors
> from affinity to field type and vice versa. Then, step-by-step we
> are replacing affinity usages with field type. Non-trivial part of
> this patch-set is removing numeric affinity (see third patch in
> series and commit message).
>
> It worth noting that current patch-set also fixes issue connected
> with invisible to query optimizer indexes, which are created from
> Lua: to choose suitable index QO checked affinity compatibility of
> key part and field def. As a result, indexes created without affinity
> field couldn't be used to process query.
>
> Nikita Pettik (8):
> sql: remove SQLITE_ENABLE_UPDATE_DELETE_LIMIT define
> sql: use field type instead of affinity for type_def
> sql: remove numeric affinity
> sql: replace affinity with field type for func
> sql: replace field type with affinity for VDBE runtime
> sql: replace affinity with field type in struct Expr
> sql: clean-up affinity from SQL source code
> Remove affinity from field definition
I've checked your patch set into 2.1 branch.
--
Regards, Kirill Yukhin
^ permalink raw reply [flat|nested] 48+ messages in thread
