From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 59C29203D7 for ; Wed, 4 Sep 2019 10:14:14 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QcG8sUQosMyT for ; Wed, 4 Sep 2019 10:14:14 -0400 (EDT) Received: from smtpng2.m.smailru.net (smtpng2.m.smailru.net [94.100.179.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 8AA4C203BE for ; Wed, 4 Sep 2019 10:14:13 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: [tarantool-patches] Re: [PATCH] sql: add check for absence in From: Roman Khabibov In-Reply-To: <20190829175955.GA52650@tarantool.org> Date: Wed, 4 Sep 2019 17:14:10 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <0FDC0655-79B5-479F-8FAA-5E9A2E93F1CF@tarantool.org> References: <20190802125252.54621-1-roman.habibov@tarantool.org> <6B834CCD-6A77-42D0-8AAD-B1BD2B8F51A6@tarantool.org> <8BD992A5-0A74-4E60-A239-5FDE85783467@tarantool.org> <326482ED-8190-4B72-B2B8-F20763A86F7E@tarantool.org> <1B494CAB-704F-4A62-A00D-257388DDB2B8@tarantool.org> <20190829175955.GA52650@tarantool.org> Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: Nikita Pettik > On Aug 29, 2019, at 20:59, Nikita Pettik = wrote: >=20 > On Wed, Aug 28, 2019 at 03:17:41PM +0300, Roman Khabibov wrote: >>=20 >>> On Aug 20, 2019, at 22:41, n.pettik wrote: >>> You slightly misunderstood me. I proposed to allow such queries >>> gracefully handling WITH statement and avoiding references >>> counter incrementation for CTEs. >>=20 >> commit 202677704c455208d6def08d373776cfae82fdbe >> Author: Roman Khabibov >> Date: Mon Jul 29 18:00:34 2019 +0300 >>=20 >> sql: allow to create view as clause >>=20 >> Allow views to use CTEs in clauses, which can be in any >> (nested) select after . >=20 > Please, provide decent commit message which would include problem > explanation, chosen solution, some of implementation details. Done. >> diff --git a/src/box/alter.cc b/src/box/alter.cc >> index 92f1d5b22..d3d6770cf 100644 >> --- a/src/box/alter.cc >> +++ b/src/box/alter.cc >> @@ -1721,6 +1721,8 @@ update_view_references(struct Select *select, = int update_value, >> const char *space_name =3D sql_src_list_entry_name(list, = i); >> if (space_name =3D=3D NULL) >> continue; >> + if (is_cte(select, space_name) =3D=3D true) >> + continue; >=20 > If function returns boolean value, you don't need to use comparison > to check it: >=20 > 'if (returns_boolean()) ...' or 'if (! returns_boolean()) ...' >=20 > Also, put comment explaining why do you skip reference counter > increment in case of SELECT containts CTE. @@ -1759,6 +1759,13 @@ update_view_references(struct Select *select, int = update_value, const char *space_name =3D sql_src_list_entry_name(list, = i); if (space_name =3D=3D NULL) continue; + /* + * CTE tables isn't held in space cache and we + * don't need to increment reference counter for + * CTEs. + */ + if (sql_select_has_cte_with_name(select, space_name)) + continue; struct space *space =3D space_by_name(space_name); if (space =3D=3D NULL) { if (! suppress_error) { >> diff --git a/src/box/sql.h b/src/box/sql.h >> index 9ccecf28c..c180e40e1 100644 >> --- a/src/box/sql.h >> +++ b/src/box/sql.h >> @@ -323,6 +323,17 @@ sql_select_delete(struct sql *db, struct Select = *select); >> struct SrcList * >> sql_select_expand_from_tables(struct Select *select); >>=20 >> +/** >> + * Check if @a name matches with at least one of CTE names typed >> + * in clauses within @a select. >> + * >> + * @param select Select to be checked. >=20 > -> Select which may contain CTE. >=20 >> + * @retval true Has CTE with @a name. >> + * @retval false Hasn't CTE with @a name. >> +*/ >> +bool >> +is_cte(struct Select *select, const char *name); >> + >> /** >> * Temporary getter in order to avoid including sqlInt.h >> * in alter.cc. >> diff --git a/src/box/sql/select.c b/src/box/sql/select.c >> index c312f61f1..0ee840b89 100644 >> --- a/src/box/sql/select.c >> +++ b/src/box/sql/select.c >> @@ -332,6 +332,25 @@ sql_select_expand_from_tables(struct Select = *select) >> return walker.u.pSrcList; >> } >>=20 >> +bool >> +is_cte(struct Select *select, const char *name) >=20 > Such a bad name. Please, get in touch with our naming policy > and come up with better one. diff --git a/src/box/sql.h b/src/box/sql.h index 7644051b4..422c8dfc2 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -323,6 +323,17 @@ sql_select_delete(struct sql *db, struct Select = *select); struct SrcList * sql_select_expand_from_tables(struct Select *select); =20 +/** + * Check if @a name matches with at least one of CTE names typed + * in clauses within @a select. + * + * @param select Select which may contain CTE. + * @retval true Has CTE with @a name. + * @retval false Hasn't CTE with @a name. +*/ +bool +sql_select_has_cte_with_name(struct Select *select, const char *name); + >> +{ >> + assert(select !=3D NULL && name !=3D NULL); >> + struct With *with =3D select->pWith; >> + if (with !=3D NULL) { >> + if (memcmp(name, with->a->zName, strlen(name)) =3D=3D 0) >> + return true; >> + } >> + struct SrcList *list =3D select->pSrc; >> + int item_count =3D sql_src_list_entry_count(list); >> + for (int i =3D 0; i < item_count; ++i) { >> + if (list->a[i].pSelect !=3D NULL) >> + if (is_cte(list->a[i].pSelect, name) =3D=3D = true) >> + return true; >> + } >=20 > It's wrong way of CTE traversal. See sqlTreeViewWith() for correct > implementation. With your approach this example still fails: >=20 > create view v as WITH RECURSIVE > xaxis(x) AS (VALUES(-2.0) UNION ALL SELECT x+0.05 FROM xaxis WHERE = x<1.2), > yaxis(y) AS (VALUES(-1.0) UNION ALL SELECT y+0.1 FROM yaxis WHERE = y<1.0), > m(iter, cx, cy, x, y) AS ( > SELECT 0, x, y, 0.0, 0.0 FROM xaxis, yaxis > UNION ALL > SELECT iter+1, cx, cy, x*x-y*y + cx, 2.0*x*y + cy FROM m=20 > WHERE (x*x + y*y) < 4.0 AND iter<28 > ), > m2(iter, cx, cy) AS ( > SELECT max(iter), cx, cy FROM m GROUP BY cx, cy > ), > a(t) AS ( > SELECT group_concat( substr(' .+*#', 1+least(iter/7,4), 1), '')=20 > FROM m2 GROUP BY cy > ) > SELECT group_concat(trim(t),x'0a') FROM a; +bool +sql_select_has_cte_with_name(struct Select *select, const char *name) +{ + assert(select !=3D NULL && name !=3D NULL); + struct With *with =3D select->pWith; + if (with !=3D NULL) { + for (int i =3D 0; i < with->nCte; i++) { + const struct Cte *cte =3D &with->a[i]; + if (memcmp(name, cte->zName, strlen(name)) =3D=3D = 0) + return true; + } + } + struct SrcList *list =3D select->pSrc; + int item_count =3D sql_src_list_entry_count(list); + for (int i =3D 0; i < item_count; ++i) { + if (list->a[i].pSelect !=3D NULL) + if = (sql_select_has_cte_with_name(list->a[i].pSelect, + name)) + return true; + } + return false; +} + >> diff --git a/test/sql-tap/view.test.lua b/test/sql-tap/view.test.lua >> index 101f4c3e7..4b695fa6a 100755 >> --- a/test/sql-tap/view.test.lua >> +++ b/test/sql-tap/view.test.lua >> @@ -1233,4 +1233,99 @@ test:do_catchsql_test( >> -- >> }) >>=20 >> +-- gh-4149: Check error message for view creation with (nested) >> +-- select with clause. >=20 > -> gh-4149: make sure that VIEW can be created as CTE. >=20 >> +test:do_execsql_test( >> + "view-24.1", >> + [[ >> + CREATE TABLE ts (s1 INT PRIMARY KEY); >> + INSERT INTO ts VALUES (1); >> + ]], { >> + -- >> + -- >> + }) >> + >> +test:do_execsql_test( >> + "view-24.2", >> + [[ >> + CREATE VIEW v AS WITH w(id) AS ( >> + SELECT 1) >> + SELECT * FROM ts; >> + ]], { >=20 > Please test not only the fact that view can be created, but also that > it is queryable (i.e. SELECT * FROM v is processed without accidents). >=20 >> + -- >> + -- >> + }) >>=20 >=20 > All examples below are almost identical since nCte for them equals to = 1. >=20 >> +test:do_execsql_test( >> + "view-24.3", >> + [[ >> + DROP VIEW v; >> + CREATE VIEW v AS WITH RECURSIVE w AS ( >> + SELECT s1 FROM ts >> + UNION ALL >> + SELECT s1+1 FROM w WHERE s1 < 4) >> + SELECT * FROM w; >> + ]], { >> + -- >> + -- >> + }) >> + >> +test:do_execsql_test( >> + "view-24.4", >> + [[ >> + DROP VIEW v; >> + CREATE VIEW v AS SELECT * FROM ( >> + WITH RECURSIVE w AS ( >> + SELECT s1 FROM ts >> + UNION ALL >> + SELECT s1+1 FROM w WHERE s1 < 4) >> + SELECT * FROM w); >> + ]], { >> + -- >> + -- >> + }) >> + >> +test:do_execsql_test( >> + "view-24.5", >> + [[ >> + DROP VIEW v; >> + CREATE VIEW v AS SELECT * FROM ( >> + SELECT * FROM ( >> + WITH RECURSIVE w AS ( >> + SELECT s1 FROM ts >> + UNION ALL >> + SELECT s1+1 FROM w WHERE s1 < 4) >> + SELECT * FROM w)); >> + ]], { >> + -- >> + -- >> + }) >> + >> +test:do_execsql_test( >> + "view-24.6", >> + [[ >> + DROP VIEW v; >> + CREATE VIEW v AS SELECT * FROM >> + (SELECT 1), >> + (SELECT 2) JOIN >> + (WITH RECURSIVE w AS ( >> + SELECT s1 FROM ts >> + UNION ALL >> + SELECT s1+1 FROM w WHERE s1 < 4) >> + SELECT * FROM w); >> + ]], { >> + -- >> + -- >> + }) >> + >> +test:do_execsql_test( >> + "view-24.7", >> + [[ >> + DROP VIEW v; >> + DROP TABLE ts; >=20 > You don't have to provide clean-up in SQL-tap suite. I have reduced the number of tests. Why? I make sure that nested select are handled too in these tests. +-- gh-4149: Make sure that VIEW can be created as CTE. +test:do_execsql_test( + "view-24.1", + [[ + CREATE TABLE ts (s1 INT PRIMARY KEY); + INSERT INTO ts VALUES (1); + ]], { + -- + -- + }) + +test:do_execsql_test( + "view-24.2", + [[ + CREATE VIEW v AS WITH w(id) AS ( + SELECT 1) + SELECT * FROM ts; + SELECT * FROM v; + ]], { + -- + 1 + -- + }) + +test:do_execsql_test( + "view-24.3", + [[ + DROP VIEW v; + CREATE VIEW v AS WITH RECURSIVE w AS ( + SELECT s1 FROM ts + UNION ALL + SELECT s1+1 FROM w WHERE s1 < 4) + SELECT * FROM w; + SELECT * FROM v; + ]], { + -- + 1,2,3,4 + -- + }) + +test:do_execsql_test( + "view-24.4", + [[ + DROP VIEW v; + CREATE VIEW v AS SELECT * FROM + (SELECT 1), + (SELECT 2) JOIN + (WITH RECURSIVE w AS ( + SELECT s1 FROM ts + UNION ALL + SELECT s1+1 FROM w WHERE s1 < 4) + SELECT * FROM w); + SELECT * FROM v; + ]], { + -- + 1,2,1,1,2,2,1,2,3,1,2,4 + -- + }) + +test:do_execsql_test( + "view-24.5", + [[ + DROP VIEW v; + DROP TABLE ts; + CREATE VIEW v AS WITH RECURSIVE + xaxis(x) AS ( + VALUES(-2.0) + UNION ALL + SELECT x+0.5 FROM xaxis WHERE x<1.2), + yaxis(y) AS ( + VALUES(-1.0) + UNION ALL + SELECT y+0.5 FROM yaxis WHERE y<1.0), + m(iter, cx, cy, x, y) AS ( + SELECT 0, x, y, 0.0, 0.0 FROM xaxis, yaxis + UNION ALL + SELECT iter+1, cx, cy, x*x-y*y + cx, 2.0*x*y + cy FROM = m WHERE (x*x + y*y) < 4.0 AND iter<1 + ), + m2(iter, cx, cy) AS ( + SELECT max(iter), cx, cy FROM m GROUP BY cx, cy + ), + a(t) AS ( + SELECT group_concat( substr('a', 1+least(iter/7,4), 1), = '') FROM m2 GROUP BY cy + ) + SELECT group_concat(trim(t),x'0a') FROM a; + SELECT * FROM v; + ]], { + -- + "aaaaaaaa\naaaaaaaa\naaaaaaaa\naaaaaaaa\naaaaaaaa" + -- + }) + commit 4f78ff7cf8b4a3f496d72388235f773ae51d6efc Author: Roman Khabibov Date: Mon Jul 29 18:00:34 2019 +0300 sql: allow to create view as clause =20 Allow views to use CTEs in clauses, which can be in any (nested) select after . Before this patch, during view creation all referenced spaces were fetched by name from SELECT and their reference counters were incremented to avoid dangling references. It occurred in update_view_references(). Obviously, CTE tables weren't held in space cache, ergo error "space doesn=E2=80=99t exist" was raised. Add check if a space from FROM is CTE. If it is, don't increment its reference counter and don't raise the error. =20 Closes #4149 diff --git a/src/box/alter.cc b/src/box/alter.cc index 4f2e34bf0..a625d8c39 100644 --- a/src/box/alter.cc +++ b/src/box/alter.cc @@ -1759,6 +1759,13 @@ update_view_references(struct Select *select, int = update_value, const char *space_name =3D sql_src_list_entry_name(list, = i); if (space_name =3D=3D NULL) continue; + /* + * CTE tables isn't held in space cache and we + * don't need to increment reference counter for + * CTEs. + */ + if (sql_select_has_cte_with_name(select, space_name)) + continue; struct space *space =3D space_by_name(space_name); if (space =3D=3D NULL) { if (! suppress_error) { diff --git a/src/box/sql.h b/src/box/sql.h index 7644051b4..422c8dfc2 100644 --- a/src/box/sql.h +++ b/src/box/sql.h @@ -323,6 +323,17 @@ sql_select_delete(struct sql *db, struct Select = *select); struct SrcList * sql_select_expand_from_tables(struct Select *select); =20 +/** + * Check if @a name matches with at least one of CTE names typed + * in clauses within @a select. + * + * @param select Select which may contain CTE. + * @retval true Has CTE with @a name. + * @retval false Hasn't CTE with @a name. +*/ +bool +sql_select_has_cte_with_name(struct Select *select, const char *name); + /** * Temporary getter in order to avoid including sqlInt.h * in alter.cc. diff --git a/src/box/sql/select.c b/src/box/sql/select.c index ddb5de87e..d7dab35cd 100644 --- a/src/box/sql/select.c +++ b/src/box/sql/select.c @@ -332,6 +332,29 @@ sql_select_expand_from_tables(struct Select = *select) return walker.u.pSrcList; } =20 +bool +sql_select_has_cte_with_name(struct Select *select, const char *name) +{ + assert(select !=3D NULL && name !=3D NULL); + struct With *with =3D select->pWith; + if (with !=3D NULL) { + for (int i =3D 0; i < with->nCte; i++) { + const struct Cte *cte =3D &with->a[i]; + if (memcmp(name, cte->zName, strlen(name)) =3D=3D = 0) + return true; + } + } + struct SrcList *list =3D select->pSrc; + int item_count =3D sql_src_list_entry_count(list); + for (int i =3D 0; i < item_count; ++i) { + if (list->a[i].pSelect !=3D NULL) + if = (sql_select_has_cte_with_name(list->a[i].pSelect, + name)) + return true; + } + return false; +} + /* * Given 1 to 3 identifiers preceding the JOIN keyword, determine the * type of join. Return an integer constant that expresses that type diff --git a/test/sql-tap/view.test.lua b/test/sql-tap/view.test.lua index 101f4c3e7..67e068b42 100755 --- a/test/sql-tap/view.test.lua +++ b/test/sql-tap/view.test.lua @@ -1,6 +1,6 @@ #!/usr/bin/env tarantool test =3D require("sqltester") -test:plan(73) +test:plan(78) =20 --!./tcltestrunner.lua -- 2002 February 26 @@ -1233,4 +1233,96 @@ test:do_catchsql_test( -- }) =20 +-- gh-4149: Make sure that VIEW can be created as CTE. +test:do_execsql_test( + "view-24.1", + [[ + CREATE TABLE ts (s1 INT PRIMARY KEY); + INSERT INTO ts VALUES (1); + ]], { + -- + -- + }) + +test:do_execsql_test( + "view-24.2", + [[ + CREATE VIEW v AS WITH w(id) AS ( + SELECT 1) + SELECT * FROM ts; + SELECT * FROM v; + ]], { + -- + 1 + -- + }) + +test:do_execsql_test( + "view-24.3", + [[ + DROP VIEW v; + CREATE VIEW v AS WITH RECURSIVE w AS ( + SELECT s1 FROM ts + UNION ALL + SELECT s1+1 FROM w WHERE s1 < 4) + SELECT * FROM w; + SELECT * FROM v; + ]], { + -- + 1,2,3,4 + -- + }) + +test:do_execsql_test( + "view-24.4", + [[ + DROP VIEW v; + CREATE VIEW v AS SELECT * FROM + (SELECT 1), + (SELECT 2) JOIN + (WITH RECURSIVE w AS ( + SELECT s1 FROM ts + UNION ALL + SELECT s1+1 FROM w WHERE s1 < 4) + SELECT * FROM w); + SELECT * FROM v; + ]], { + -- + 1,2,1,1,2,2,1,2,3,1,2,4 + -- + }) + +test:do_execsql_test( + "view-24.5", + [[ + DROP VIEW v; + DROP TABLE ts; + CREATE VIEW v AS WITH RECURSIVE + xaxis(x) AS ( + VALUES(-2.0) + UNION ALL + SELECT x+0.5 FROM xaxis WHERE x<1.2), + yaxis(y) AS ( + VALUES(-1.0) + UNION ALL + SELECT y+0.5 FROM yaxis WHERE y<1.0), + m(iter, cx, cy, x, y) AS ( + SELECT 0, x, y, 0.0, 0.0 FROM xaxis, yaxis + UNION ALL + SELECT iter+1, cx, cy, x*x-y*y + cx, 2.0*x*y + cy FROM = m WHERE (x*x + y*y) < 4.0 AND iter<1 + ), + m2(iter, cx, cy) AS ( + SELECT max(iter), cx, cy FROM m GROUP BY cx, cy + ), + a(t) AS ( + SELECT group_concat( substr('a', 1+least(iter/7,4), 1), = '') FROM m2 GROUP BY cy + ) + SELECT group_concat(trim(t),x'0a') FROM a; + SELECT * FROM v; + ]], { + -- + "aaaaaaaa\naaaaaaaa\naaaaaaaa\naaaaaaaa\naaaaaaaa" + -- + }) + test:finish_test()