From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id C3169267B2 for ; Tue, 31 Jul 2018 07:48:19 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W-BJBcIqfNz1 for ; Tue, 31 Jul 2018 07:48:19 -0400 (EDT) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id EE9462673E for ; Tue, 31 Jul 2018 07:48:18 -0400 (EDT) Received: by mail-lf1-f49.google.com with SMTP id g6-v6so10487078lfb.11 for ; Tue, 31 Jul 2018 04:48:18 -0700 (PDT) MIME-Version: 1.0 References: <8B8D5501-075D-4BEB-B282-35B0B81CD555@tarantool.org> <605B15EF-BD1C-4B03-8A9F-6E6225076812@tarantool.org> <12B62C73-9BEC-49FA-B3FD-590C445CF25B@tarantool.org> <2123605D-8D6C-43A3-846F-735E4C2C7FC2@tarantool.org> <20180729011251.eitp7cisv6jv5opj@tkn_work_nb> In-Reply-To: <20180729011251.eitp7cisv6jv5opj@tkn_work_nb> From: Nikita Tatunov Date: Tue, 31 Jul 2018 14:48:04 +0300 Message-ID: Subject: [tarantool-patches] Re: [PATCH] sql: xfer optimization issue Content-Type: multipart/alternative; boundary="0000000000005c49ca05724a2778" Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-subscribe: List-owner: List-post: List-archive: To: Alexander Turenko Cc: tarantool-patches@freelists.org, korablev@tarantool.org --0000000000005c49ca05724a2778 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =D0=B2=D1=81, 29 =D0=B8=D1=8E=D0=BB. 2018 =D0=B3. =D0=B2 4:12, Alexander Tu= renko < alexander.turenko@tarantool.org>: > Hi! > > Please consider my comments and questions below. > > WBR, Alexander Turenko. > > > + /* > > + * Xfer optimization is unable to correctly insert data > > + * in case there's a conflict action other than > > + * explicit *_ABORT. This is the reason we want to only > > + * run it if the destination table is initially empty. > > + * That block generates code to make that determination. > > + */ > > + if (!(onError =3D=3D ON_CONFLICT_ACTION_ABORT && > > + !is_err_action_default)) { > > Do you mean that: > > 1. The optimization non-empty table case correctly works only with ABORT > conflict action (default or explicit). > 2. The default conflict action can be overwritten on per-column basis, so > 'default abort' can really be replace or other conflict action. > > If so, your description doesn't give this information. > > To more on that, can we check per-column conflict actions instead of chec= k > whether the conflict action default or explicit? This would enable more > cases > with non-empty tables for the optimization. And this would look less > confusing, > IMHO. > Well, basically, you're right. But the thing is that we're going to remove column conflict actions, thus, I suppose, it doesn't make sense. Nikita, Correct me if I'm wrong please. > > I have one more question on that. It seems that SQLite has this > optimization > working with ROLLBACK conflict action. We cannot doing so, because of som= e > problem? Did this problem described / trackerized somewhere? Or something > changes underhood and makes this impossible? Are we know exact reason or > just > observing it does not work? > I investigated that a little. OP_IdxInsert was changed so that we can process ABORT, FAIL or IGNORE. I took it into account in newer version hence it is also used in these cases even when the destination table is not empty= . > > > +#ifdef SQLITE_TEST > > +/* > > + * The following global variable is incremented whenever the > > + * transfer optimization is used. This is used for testing > > + * purposes only - to make sure the transfer optimization really > > + * is happening when it is supposed to. > > + */ > > +int sql_xfer_count =3D 0; > > +#endif > > I think it would be good to mention the opcode where the counter is > incremented. You can follow the style in which other counters are > described (they are mostly mention opcodes). > Added. > > > -/* Opcode: RowData P1 P2 * * * > > +/* Opcode: RowData P1 P2 * * P5 > > We can increase the counter on per-operation basis instead of per-row by > adding the flag to OP_OpenWrite. It will save some CPU cycles :) > > +#ifdef SQLITE_TEST > > + if ((pOp->p5 & OPFLAG_XFER_OPT) !=3D 0) { > > + pOp->p5 =3D 0; > > + sql_xfer_count++; > > + } > > +#endif > > 1. Not actual due to 2, but it would be better to use > `pOp->p5 &=3D ~OPFLAG_XFER_OPT` to drop just that flag. > 2. It is counter-intuitive, IMHO, to change operation flags during that > operation. So, said above, vote to move it to OP_OpenWrite. > > Made the first. > > +local bfr, aftr > > + > > What do you plan to do with saved letters? :) Really, such abbreviations > just makes reading harder with no gains. > I made dis :( (p.s. Changed it) > > > +local function do_xfer_test(test_number, return_code) > > + test_name =3D string.format("xfer-optimization-1.%d", test_numb= er) > > + test:do_test( > > + test_name, > > + function() > > + return {aftr - bfr} > > + end, { > > + -- > > + return_code > > + -- > > + }) > > +end > > That code can be written simpler (consider tap module documentation): > > test:is(after - before, exp, test_name) > > I suggest to create wrappers like so (I didn't test it): > > local function do_xfer_test(test_func, test, test_name, func, exp, opts) > local opts =3D opts or {} > local exp_xfer_count =3D opts.exp_xfer_count > local before =3D box.sql.debug().sql_xfer_count > local ok =3D test_func(test, test_name, func, exp) > local after =3D box.sql.debug().sql_xfer_count > if exp_xfer_count ~=3D nil then > ok =3D ok and test:is(after - before, exp_xfer_count, test_name .= . > '_xfer_count') > end > return ok > end > > test.do_execsql_xfer_test =3D function(test, test_name, func, exp, opts) > return do_xfer_test(test.do_execsql_test, test, test_name, func, exp, > opts) > end > > test.do_catchsql_xfer_test =3D function(test, test_name, func, exp, opts) > return do_xfer_test(test.do_catchsql_test, test, test_name, func, exp= , > opts) > end > > And use it like so: > > test:do_catchsql_xfer_test( > "xfer-optimization-1.1", > [[ > CREATE TABLE t1(a INTEGER PRIMARY KEY, b INTEGER UNIQUE); > INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3); > CREATE TABLE t2(a INTEGER PRIMARY KEY, b INTEGER UNIQUE); > INSERT INTO t2 SELECT * FROM t1; > ]], { > -- > 0 > -- > }), { > exp_xfer_count =3D 1 > } > ) > > Thank you for suggestion. Added that implementation. (Needed to change it a little bit) > By the way, you can revive xfer cases in test/sql-tap/with2.test.lua. Or > drop > it if your new test includes all related cases from with2. > > They're alive now! > On Fri, Jul 20, 2018 at 07:58:48PM +0300, Nikita Tatunov wrote: > > Ooops. Thank you! fixed it and pushed. > > > > =D0=BF=D1=82, 20 =D0=B8=D1=8E=D0=BB. 2018 =D0=B3. =D0=B2 19:43, n.pe= ttik <[1]korablev@tarantool.org>: > > > > LGTM. > > > > diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c > > index 3c3bf37..4f52fa5 100644 > > --- a/src/box/sql/insert.c > > +++ b/src/box/sql/insert.c > > @@ -1869,7 +1869,7 @@ xferOptimization(Parse * pParse, /* Parser > > context */ > > * table (tab1) is initially empty. > > */ > > > > - /* The Vdbe we're building*/ > > + /* The Vdbe struct we're building. */ > > > > You misunderstood me. What I mean is: > > struct Vibe *v =3D =E2=80=A6; > > > > Vdbe *v =3D sqlite3GetVdbe(pParse); > > > > References > > > > 1. mailto:korablev@tarantool.org Diff: diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c index 3c45920..c3cf94a 100644 --- a/src/box/sql/insert.c +++ b/src/box/sql/insert.c @@ -1715,7 +1715,6 @@ xferOptimization(Parse * pParse, /* Parser context */ int iSrc, iDest; /* Cursors from source and destination */ int addr1; /* Loop addresses */ int emptyDestTest =3D 0; /* Address of test for empty pDest */ - int emptySrcTest =3D 0; /* Address of test for empty pSrc */ int regData, regTupleid; /* Registers holding data and tupleid */ struct session *user_session =3D current_session(); bool is_err_action_default =3D false; @@ -1881,13 +1880,15 @@ xferOptimization(Parse * pParse, /* Parser context */ /* * Xfer optimization is unable to correctly insert data - * in case there's a conflict action other than - * explicit *_ABORT. This is the reason we want to only + * in case there's a conflict action other than *_ABORT, + * *_FAIL or *_IGNORE. This is the reason we want to only * run it if the destination table is initially empty. * That block generates code to make that determination. */ - if (!(onError =3D=3D ON_CONFLICT_ACTION_ABORT && - !is_err_action_default)) { + if (!(onError =3D=3D ON_CONFLICT_ACTION_ABORT || + onError =3D=3D ON_CONFLICT_ACTION_FAIL || + onError =3D=3D ON_CONFLICT_ACTION_IGNORE) || + is_err_action_default) { addr1 =3D sqlite3VdbeAddOp2(v, OP_Rewind, iDest, 0); VdbeCoverage(v); emptyDestTest =3D sqlite3VdbeAddOp0(v, OP_Goto); @@ -1905,15 +1906,23 @@ xferOptimization(Parse * pParse, /* Parser context */ #endif sqlite3VdbeAddOp2(v, OP_IdxInsert, iDest, regData); - sqlite3VdbeChangeP5(v, OPFLAG_NCHANGE); + switch (onError) { + case ON_CONFLICT_ACTION_IGNORE: + sqlite3VdbeChangeP5(v, OPFLAG_OE_IGNORE); + break; + case ON_CONFLICT_ACTION_FAIL: + sqlite3VdbeChangeP5(v, OPFLAG_OE_FAIL); + break; + default: + sqlite3VdbeChangeP5(v, OPFLAG_NCHANGE); + break; + } sqlite3VdbeAddOp2(v, OP_Next, iSrc, addr1 + 1); VdbeCoverage(v); sqlite3VdbeJumpHere(v, addr1); sqlite3VdbeAddOp2(v, OP_Close, iSrc, 0); sqlite3VdbeAddOp2(v, OP_Close, iDest, 0); - if (emptySrcTest) - sqlite3VdbeJumpHere(v, emptySrcTest); sqlite3ReleaseTempReg(pParse, regTupleid); sqlite3ReleaseTempReg(pParse, regData); if (emptyDestTest) { diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index aa7f250..ce01039 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -79,10 +79,10 @@ int sql_search_count =3D 0; #ifdef SQLITE_TEST /* - * The following global variable is incremented whenever the - * transfer optimization is used. This is used for testing - * purposes only - to make sure the transfer optimization really - * is happening when it is supposed to. + * The following global variable is incremented in OP_RowData + * whenever the xfer optimization is used. This is used on + * testing purposes only - to make sure the transfer optimization + * really is happening when it is supposed to. */ int sql_xfer_count =3D 0; #endif @@ -4008,9 +4008,13 @@ case OP_RowData: { BtCursor *pCrsr; u32 n; +/* + * Flag P5 is cleared after the first insertion using xfer + * optimization. + */ #ifdef SQLITE_TEST if ((pOp->p5 & OPFLAG_XFER_OPT) !=3D 0) { - pOp->p5 =3D 0; + pOp->p5 &=3D ~OPFLAG_XFER_OPT; sql_xfer_count++; } #endif diff --git a/test/sql-tap/gh-3307-xfer-optimization-issue.test.lua b/test/sql-tap/gh-3307-xfer-optimization-issue.test.lua index 05d30c6..b99de2b 100755 --- a/test/sql-tap/gh-3307-xfer-optimization-issue.test.lua +++ b/test/sql-tap/gh-3307-xfer-optimization-issue.test.lua @@ -2,24 +2,28 @@ test =3D require("sqltester") test:plan(46) -local bfr, aftr - -local function do_xfer_test(test_number, return_code) - test_name =3D string.format("xfer-optimization-1.%d", test_number) - test:do_test( - test_name, - function() - return {aftr - bfr} - end, { - -- - return_code - -- - }) +local function do_xfer_test(test, test_func, test_name, func, exp, opts) + local opts =3D opts or {} + local exp_xfer_count =3D opts.exp_xfer_count + local before =3D box.sql.debug().sql_xfer_count + local ok, result =3D pcall(test_func, test, test_name, func, exp) + local after =3D box.sql.debug().sql_xfer_count + if exp_xfer_count ~=3D nil then + ok =3D ok and test:is(after - before, exp_xfer_count, + test_name .. '-xfer-count') + end + return ok end -bfr =3D box.sql.debug().sql_xfer_count +test.do_execsql_xfer_test =3D function(test, test_name, func, exp, opts) + return do_xfer_test(test, test.do_execsql_test, test_name, func, exp, opts) +end + +test.do_catchsql_xfer_test =3D function(test, test_name, func, exp, opts) + return do_xfer_test(test, test.do_catchsql_test, test_name, func, exp, opts) +end -test:do_catchsql_test( +test:do_catchsql_xfer_test( "xfer-optimization-1.1", [[ CREATE TABLE t1(a INTEGER PRIMARY KEY, b INTEGER UNIQUE); @@ -30,10 +34,10 @@ test:do_catchsql_test( -- 0 -- + }, { + exp_xfer_count =3D 1 }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( "xfer-optimization-1.2", [[ @@ -44,12 +48,8 @@ test:do_execsql_test( -- }) -do_xfer_test(3, 1) - -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.4", +test:do_catchsql_xfer_test( + "xfer-optimization-1.3", [[ DROP TABLE t1; DROP TABLE t2; @@ -60,15 +60,15 @@ test:do_catchsql_test( INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 1 }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( - "xfer-optimization-1.5", + "xfer-optimization-1.4", [[ SELECT * FROM t2; ]], { @@ -77,12 +77,8 @@ test:do_execsql_test( -- }) -do_xfer_test(6, 1) - -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.7", +test:do_catchsql_xfer_test( + "xfer-optimization-1.5", [[ DROP TABLE t1; DROP TABLE t2; @@ -91,29 +87,25 @@ test:do_catchsql_test( CREATE TABLE t2(a INTEGER PRIMARY KEY, b INTEGER); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 1, "table T2 has 2 columns but 3 values were supplied" - -- + -- + }, { + exp_xfer_count =3D 0 }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( - "xfer-optimization-1.8", + "xfer-optimization-1.6", [[ SELECT * FROM t2; ]], { - -- + -- - -- + -- }) -do_xfer_test(9, 0) - -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.10", +test:do_catchsql_xfer_test( + "xfer-optimization-1.7", [[ DROP TABLE t1; DROP TABLE t2; @@ -122,29 +114,25 @@ test:do_catchsql_test( CREATE TABLE t2(a INTEGER PRIMARY KEY, b INTEGER); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 1 }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( - "xfer-optimization-1.11", + "xfer-optimization-1.8", [[ SELECT * FROM t2; ]], { - -- + -- 1, 1, 2, 2, 3, 3 - -- + -- }) -do_xfer_test(12, 1); - -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.13", +test:do_catchsql_xfer_test( + "xfer-optimization-1.9", [[ DROP TABLE t1; DROP TABLE t2; @@ -153,29 +141,25 @@ test:do_catchsql_test( CREATE TABLE t2(b INTEGER, a INTEGER PRIMARY KEY); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 1, "Duplicate key exists in unique index 'sqlite_autoindex_T2_1' in space 'T2'" - -- + -- + }, { + exp_xfer_count =3D 0 }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( - "xfer-optimization-1.14", + "xfer-optimization-1.10", [[ SELECT * FROM t2; ]], { - -- + -- - -- + -- }) -do_xfer_test(15, 0) - -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.16", +test:do_catchsql_xfer_test( + "xfer-optimization-1.11", [[ DROP TABLE t1; DROP TABLE t2; @@ -184,25 +168,23 @@ test:do_catchsql_test( CREATE TABLE t2(b INTEGER PRIMARY KEY, a INTEGER); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 1 }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( - "xfer-optimization-1.17", + "xfer-optimization-1.12", [[ SELECT * FROM t2; ]], { - -- + -- 1, 1, 2, 2, 3, 2 - -- + -- }) -do_xfer_test(18, 1) - -- The following tests are supposed to test if xfer-optimization is actually -- used in the given cases (if the conflict actually occurs): -- 1.0) insert w/o explicit confl. action & w/o index replace action @@ -221,10 +203,8 @@ do_xfer_test(18, 1) -- 1.0) insert w/o explicit confl. action & w/o index replace action --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.19", +test:do_catchsql_xfer_test( + "xfer-optimization-1.13", [[ DROP TABLE t1; DROP TABLE t2; @@ -236,35 +216,31 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 1, "Duplicate key exists in unique index 'sqlite_autoindex_T2_1' in space 'T2'" - -- + -- + }, { + exp_xfer_count =3D 0 }) test:do_execsql_test( - "xfer-optimization-1.20", + "xfer-optimization-1.14", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 2, 2, 3, 4, 4, 4, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(21, 0) - -- 1.1) insert w/o explicit confl. action & w/ -- index replace action & empty dest_table --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.22", +test:do_catchsql_xfer_test( + "xfer-optimization-1.15", [[ DROP TABLE t1; DROP TABLE t2; @@ -276,45 +252,41 @@ test:do_catchsql_test( INSERT INTO t3 VALUES (1); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 1 }) test:do_execsql_test( - "xfer-optimization-1.23", + "xfer-optimization-1.16", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 1, 1, 3, 3, 5, 5, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - test:do_execsql_test( - "xfer-optimization-1.24", + "xfer-optimization-1.17", [[ SELECT * FROM t3; ]], { - -- + -- 1 - -- + -- }) -do_xfer_test(25, 1) - -- 1.2) insert w/o explicit confl. action & w/ -- index replace action & non-empty dest_table --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.26", +test:do_catchsql_xfer_test( + "xfer-optimization-1.18", [[ DROP TABLE t1; DROP TABLE t2; @@ -327,34 +299,30 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 0 }) test:do_execsql_test( - "xfer-optimization-1.27", + "xfer-optimization-1.19", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(28, 0) - -- 2) insert with abort --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.29", +test:do_catchsql_xfer_test( + "xfer-optimization-1.20", [[ DROP TABLE t1; DROP TABLE t2; @@ -366,34 +334,30 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT OR ABORT INTO t2 SELECT * FROM t1; ]], { - -- + -- 1, "Duplicate key exists in unique index 'sqlite_autoindex_T2_1' in space 'T2'" - -- + -- + }, { + exp_xfer_count =3D 1 }) test:do_execsql_test( - "xfer-optimization-1.30", + "xfer-optimization-1.21", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 2, 2, 3, 4, 4, 4, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(31, 1) - -- 3.0) insert with rollback (into empty table) --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.32", +test:do_catchsql_xfer_test( + "xfer-optimization-1.22", [[ DROP TABLE t1; DROP TABLE t2; @@ -403,34 +367,30 @@ test:do_catchsql_test( BEGIN; INSERT OR ROLLBACK INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 1 }) test:do_execsql_test( - "xfer-optimization-1.33", + "xfer-optimization-1.23", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 1, 1, 3, 3, 5, 5, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(34, 1) - -- 3.1) insert with rollback (into non-empty table) --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.35", +test:do_catchsql_xfer_test( + "xfer-optimization-1.24", [[ DROP TABLE t1; DROP TABLE t2; @@ -442,32 +402,28 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT OR ROLLBACK INTO t2 SELECT * FROM t1; ]], { - -- + -- 1, "UNIQUE constraint failed: T2.A" - -- + -- + }, { + exp_xfer_count =3D 0 }) test:do_execsql_test( - "xfer-optimization-1.36", + "xfer-optimization-1.25", [[ SELECT * FROM t2; ]], { - -- + -- 2, 2, 3, 4 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(37, 0) - -- 4) insert with replace --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.38", +test:do_catchsql_xfer_test( + "xfer-optimization-1.26", [[ DROP TABLE t1; DROP TABLE t2; @@ -479,34 +435,30 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT OR REPLACE INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 0 }) test:do_execsql_test( - "xfer-optimization-1.39", + "xfer-optimization-1.27", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(40, 0) - -- 5) insert with fail --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.41", +test:do_catchsql_xfer_test( + "xfer-optimization-1.28", [[ DROP TABLE t1; DROP TABLE t2; @@ -518,34 +470,30 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT OR FAIL INTO t2 SELECT * FROM t1; ]], { - -- + -- 1, "Duplicate key exists in unique index 'sqlite_autoindex_T2_1' in space 'T2'" - -- + -- + }, { + exp_xfer_count =3D 1 }) test:do_execsql_test( - "xfer-optimization-1.42", + "xfer-optimization-1.29", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 1, 1, 2, 2, 3, 4, 4, 4, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(43, 0) - -- 6) insert with ignore --------------------------------------------------------------------------= ---- -bfr =3D box.sql.debug().sql_xfer_count - -test:do_catchsql_test( - "xfer-optimization-1.44", +test:do_catchsql_xfer_test( + "xfer-optimization-1.30", [[ DROP TABLE t1; DROP TABLE t2; @@ -557,25 +505,23 @@ test:do_catchsql_test( INSERT INTO t2 VALUES (4, 4); INSERT OR IGNORE INTO t2 SELECT * FROM t1; ]], { - -- + -- 0 - -- + -- + }, { + exp_xfer_count =3D 1 }) test:do_execsql_test( - "xfer-optimization-1.45", + "xfer-optimization-1.31", [[ INSERT INTO t2 VALUES (10, 10); COMMIT; SELECT * FROM t2; ]], { - -- + -- 1, 1, 2, 2, 3, 4, 4, 4, 5, 5, 10, 10 - -- + -- }) -aftr =3D box.sql.debug().sql_xfer_count - -do_xfer_test(46, 0) - test:finish_test() diff --git a/test/sql-tap/with2.test.lua b/test/sql-tap/with2.test.lua index fbd1f4e..bd0187f 100755 --- a/test/sql-tap/with2.test.lua +++ b/test/sql-tap/with2.test.lua @@ -1,6 +1,6 @@ #!/usr/bin/env tarantool test =3D require("sqltester") -test:plan(44) +test:plan(59) --!./tcltestrunner.lua -- 2014 January 11 @@ -382,43 +382,136 @@ genstmt(255), { -- -- -- }) ---------------------------------------------------------------------------= --- --- Check that adding a WITH clause to an INSERT disables the xfer +----------------------------------------------------------------- +-- Check that adding a WITH clause to an INSERT disables the xfer -- optimization. --- --- Tarantool: `sqlite3_xferopt_count` is not exported --- Need to understand if this optimization works at all and if we really need it. --- Commented so far. --- function do_xfer_test(tn, bXfer, sql, res) --- res =3D res or "" --- sqlite3_xferopt_count =3D 0 --- X(267, "X!cmd", [=3D[["uplevel",[["list","do_test",["tn"],["\n s= et dres [db eval {",["sql"],"}]\n list [set ::sqlite3_xferopt_count] [set dres]\n "],[["list",["bXfer"],["res"]]]]]]]=3D]) --- end --- test:do_execsql_test( --- 5.1, --- [[ --- DROP TABLE IF EXISTS t1; --- DROP TABLE IF EXISTS t2; --- CREATE TABLE t1(a PRIMARY KEY, b); --- CREATE TABLE t2(a PRIMARY KEY, b); --- ]]) - --- do_xfer_test(5.2, 1, " INSERT INTO t1 SELECT * FROM t2 ") --- do_xfer_test(5.3, 0, " INSERT INTO t1 SELECT a, b FROM t2 ") --- do_xfer_test(5.4, 0, " INSERT INTO t1 SELECT b, a FROM t2 ") --- do_xfer_test(5.5, 0, [[ --- WITH x AS (SELECT a, b FROM t2) INSERT INTO t1 SELECT * FROM x --- ]]) --- do_xfer_test(5.6, 0, [[ --- WITH x AS (SELECT a, b FROM t2) INSERT INTO t1 SELECT * FROM t2 --- ]]) --- do_xfer_test(5.7, 0, [[ --- INSERT INTO t1 WITH x AS ( SELECT * FROM t2 ) SELECT * FROM x --- ]]) --- do_xfer_test(5.8, 0, [[ --- INSERT INTO t1 WITH x(a,b) AS ( SELECT * FROM t2 ) SELECT * FROM x --- ]]) +local function do_xfer_test(test, test_func, test_name, func, exp, opts) + local opts =3D opts or {} + local exp_xfer_count =3D opts.exp_xfer_count + local before =3D box.sql.debug().sql_xfer_count + local ok, result =3D pcall(test_func, test, test_name, func, exp) + local after =3D box.sql.debug().sql_xfer_count + if exp_xfer_count ~=3D nil then + ok =3D ok and test:is(after - before, exp_xfer_count, + test_name .. '-xfer-count') + end + return ok +end + +test.do_execsql_xfer_test =3D function(test, test_name, func, exp, opts) + return do_xfer_test(test, test.do_execsql_test, test_name, func, exp, opts) +end + +test.do_catchsql_xfer_test =3D function(test, test_name, func, exp, opts) + return do_xfer_test(test, test.do_catchsql_test, test_name, func, exp, opts) +end + +test:do_execsql_test( + 5.1, + [[ + DROP TABLE IF EXISTS t1; + DROP TABLE IF EXISTS t2; + CREATE TABLE t1(a PRIMARY KEY, b); + CREATE TABLE t2(a PRIMARY KEY, b); + INSERT INTO t2 VALUES (1, 1), (2, 2); + ]], { + -- <5.1> + + -- <5.1> + }) + +test:do_execsql_xfer_test( + 5.2, + [[ + INSERT INTO t1 SELECT * FROM t2; + DELETE FROM t1; + ]], { + -- <5.2> + + -- <5.2> + }, { + exp_xfer_count =3D 1 + }) + +test:do_execsql_xfer_test( + 5.3, + [[ + INSERT INTO t1 SELECT a, b FROM t2; + DELETE FROM t1; + ]], { + -- <5.3> + + -- <5.3> + }, { + exp_xfer_count =3D 0 + }) + +test:do_execsql_xfer_test( + 5.4, + [[ + INSERT INTO t1 SELECT b, a FROM t2; + DELETE FROM t1; + ]], { + -- <5.4> + + -- <5.4> + }, { + exp_xfer_count =3D 0 + }) + +test:do_execsql_xfer_test( + 5.5, + [[ + WITH x AS (SELECT a, b FROM t2) INSERT INTO t1 SELECT * FROM x; + DELETE FROM t1; + ]], { + -- <5.5> + + -- <5.5> + }, { + exp_xfer_count =3D 0 + }) + +test:do_execsql_xfer_test( + 5.6, + [[ + WITH x AS (SELECT a, b FROM t2) INSERT INTO t1 SELECT * FROM t2; + DELETE FROM t1; + ]], { + -- <5.6> + + -- <5.6> + }, { + exp_xfer_count =3D 0 + }) + +test:do_execsql_xfer_test( + 5.7, + [[ + INSERT INTO t1 WITH x AS (SELECT * FROM t2) SELECT * FROM x; + DELETE FROM t1; + ]], { + -- <5.7> + + -- <5.7> + }, { + exp_xfer_count =3D 0 + }) + +test:do_execsql_xfer_test( + 5.8, + [[ + INSERT INTO t1 WITH x(a,b) AS (SELECT * FROM t2) SELECT * FROM x; + DELETE FROM t1; + ]], { + -- <5.8> + + -- <5.8> + }, { + exp_xfer_count =3D 0 + }) + --------------------------------------------------------------------------= --- -- Check that syntax (and other) errors in statements with WITH clauses -- attached to them do not cause problems (e.g. memory leaks). --0000000000005c49ca05724a2778 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=D0=B2= =D1=81, 29 =D0=B8=D1=8E=D0=BB. 2018 =D0=B3. =D0=B2 4:12, Alexander Turenko = <al= exander.turenko@tarantool.org>:
Hi!

Please consider my comments and questions below.

WBR, Alexander Turenko.

> +=C2=A0 =C2=A0 =C2=A0 =C2=A0/*
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * Xfer optimization is unable to correctl= y insert data
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * in case there's a conflict action o= ther than
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * explicit *_ABORT. This is the reason we= want to only
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * run it if the destination table is init= ially empty.
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * That block generates code to make that = determination.
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!(onError =3D=3D ON_CONFLICT_ACTION_AB= ORT &&
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0!is_err_action_default)) {
Do you mean that:

1. The optimization non-empty table case correctly works only with ABORT =C2=A0 =C2=A0conflict action (default or explicit).
2. The default conflict action can be overwritten on per-column basis, so =C2=A0 =C2=A0'default abort' can really be replace or other conflic= t action.

If so, your description doesn't give this information.

To more on that, can we check per-column conflict actions instead of check<= br> whether the conflict action default or explicit? This would enable more cas= es
with non-empty tables for the optimization. And this would look less confus= ing,
IMHO.

Well, basically, you're right= . But the thing is that we're going to remove
column conflict= actions, thus, I suppose, it doesn't make sense.
Nikita, Cor= rect me if I'm wrong please.
=C2=A0

I have one more question on that. It seems that SQLite has this optimizatio= n
working with ROLLBACK conflict action. We cannot doing so, because of some<= br> problem? Did this problem described / trackerized somewhere? Or something changes underhood and makes this impossible? Are we know exact reason or ju= st
observing it does not work?

I investiga= ted that a little. OP_IdxInsert was changed so that we can process
ABORT, FAIL or IGNORE. I took it into account in newer version hence=C2= =A0
it is=C2=A0also used in these cases eve= n when the destination table is not empty.
=C2=A0

> +#ifdef SQLITE_TEST
> +/*
> + * The following global variable is incremented whenever the
> + * transfer optimization is used.=C2=A0 This is used for testing
> + * purposes only - to make sure the transfer optimization really
> + * is happening when it is supposed to.
> + */
> +int sql_xfer_count =3D 0;
> +#endif

I think it would be good to mention the opcode where the counter is
incremented. You can follow the style in which other counters are
described (they are mostly mention opcodes).

Added.
=C2=A0

> -/* Opcode: RowData P1 P2 * * *
> +/* Opcode: RowData P1 P2 * * P5

We can increase the counter on per-operation basis instead of per-row by adding the flag to OP_OpenWrite. It will save some CPU cycles :)
=C2=A0
=
> +#ifdef SQLITE_TEST
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0if ((pOp->p5 & OPFLAG_XFER_OPT) != =3D 0) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pOp->p5 =3D= 0;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sql_xfer_count= ++;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0}
> +#endif

1. Not actual due to 2, but it would be better to use
=C2=A0 =C2=A0`pOp->p5 &=3D ~OPFLAG_XFER_OPT` to drop just that flag.=
2. It is counter-intuitive, IMHO, to change operation flags during that
=C2=A0 =C2=A0operation. So, said above, vote to move it to OP_OpenWrite.

Made the first.
=C2=A0
=
> +local bfr, aftr
> +

What do you plan to do with saved letters? :) Really, such abbreviations just makes reading harder with no gains.

I made dis :(
(p.s. Changed it)
=C2=A0

> +local function do_xfer_test(test_number, return_code)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0test_name =3D string.format("xfer-opt= imization-1.%d", test_number)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0test:do_test(
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0test_name,
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0function()
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0return {aftr - bfr}
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0end, {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0-- <test_name>
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0return_code
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0-- <test_name>
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0})
> +end

That code can be written simpler (consider tap module documentation):

test:is(after - before, exp, test_name)

I suggest to create wrappers like so (I didn't test it):

local function do_xfer_test(test_func, test, test_name, func, exp, opts) =C2=A0 =C2=A0 local opts =3D opts or {}
=C2=A0 =C2=A0 local exp_xfer_count =3D opts.exp_xfer_count
=C2=A0 =C2=A0 local before =3D box.sql.debug().sql_xfer_count
=C2=A0 =C2=A0 local ok =3D test_func(test, test_name, func, exp)
=C2=A0 =C2=A0 local after =3D box.sql.debug().sql_xfer_count
=C2=A0 =C2=A0 if exp_xfer_count ~=3D nil then
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ok =3D ok and test:is(after - before, exp_xfer_= count, test_name ..
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 '_xfer_count')
=C2=A0 =C2=A0 end
=C2=A0 =C2=A0 return ok
end

test.do_execsql_xfer_test =3D function(test, test_name, func, exp, opts) =C2=A0 =C2=A0 return do_xfer_test(test.do_execsql_test, test, test_name, fu= nc, exp, opts)
end

test.do_catchsql_xfer_test =3D function(test, test_name, func, exp, opts) =C2=A0 =C2=A0 return do_xfer_test(test.do_catchsql_test, test, test_name, f= unc, exp, opts)
end

And use it like so:

test:do_catchsql_xfer_test(
=C2=A0 =C2=A0 "xfer-optimization-1.1",
=C2=A0 =C2=A0 [[
=C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATE TABLE t1(a INTEGER PRIMARY KEY, b INTEGE= R UNIQUE);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3); =C2=A0 =C2=A0 =C2=A0 =C2=A0 CREATE TABLE t2(a INTEGER PRIMARY KEY, b INTEGE= R UNIQUE);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 INSERT INTO t2 SELECT * FROM t1;
=C2=A0 =C2=A0 ]], {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 -- <xfer-optimization-1.1>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 -- <xfer-optimization-1.1>
=C2=A0 =C2=A0 }), {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 exp_xfer_count =3D 1
=C2=A0 =C2=A0 }
)


Thank you for suggestion. Added that i= mplementation.
(Needed to change it a little bit)
=C2= =A0
By the way, you can revive xfer cases in test/sql-tap/with2.test.lua. Or dr= op
it if your new test includes all related cases from with2.


They're alive now!
=C2= =A0
On Fri, Jul 20, 2018 at 07:58:48PM +0300, Nikita Tatunov wrote:
>=C2=A0 =C2=A0 Ooops. Thank you! fixed it and pushed.
>
>=C2=A0 =C2=A0 =D0=BF=D1=82, 20 =D0=B8=D1=8E=D0=BB. 2018 =D0=B3. =D0=B2 = 19:43, n.pettik <[1]korablev@tarantool.org>:
>
>=C2=A0 =C2=A0 LGTM.
>
>=C2=A0 =C2=A0 diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c<= br> >=C2=A0 =C2=A0 index 3c3bf37..4f52fa5 100644
>=C2=A0 =C2=A0 --- a/src/box/sql/insert.c
>=C2=A0 =C2=A0 +++ b/src/box/sql/insert.c
>=C2=A0 =C2=A0 @@ -1869,7 +1869,7 @@ xferOptimization(Parse * pParse, /*= Parser
>=C2=A0 =C2=A0 context */
>=C2=A0 =C2=A0 =C2=A0 =C2=A0* table (tab1) is initially empty.
>=C2=A0 =C2=A0 =C2=A0 =C2=A0*/
>
>=C2=A0 =C2=A0 - /* The Vdbe we're building*/
>=C2=A0 =C2=A0 + /* The Vdbe struct we're building. */
>
>=C2=A0 =C2=A0 You misunderstood me. What I mean is:
>=C2=A0 =C2=A0 struct Vibe *v =3D =E2=80=A6;
>
>=C2=A0 =C2=A0 =C2=A0 Vdbe *v =3D sqlite3GetVdbe(pParse);
>
> References
>
>=C2=A0 =C2=A0 1. mailto:korablev@tarantool.org

Dif= f:

=C2=A0diff --git a/src/box/sql/insert.c b/src/b= ox/sql/insert.c
index 3c45920..c3cf94a 100644
--- a/src= /box/sql/insert.c
+++ b/src/box/sql/insert.c
@@ -1715,7= +1715,6 @@ xferOptimization(Parse * pParse, /* Parser context */
=C2=A0 int iSrc, iDest; /* Curso= rs from source and destination */
=C2=A0 int addr1; /* Loop a= ddresses */
=C2=A0 int emp= tyDestTest =3D 0; /* Address of test= for empty pDest */
- int = emptySrcTest =3D 0; /* Address of te= st for empty pSrc */
=C2=A0 int regData, regTupleid; /* Regist= ers holding data and tupleid */
=C2=A0 struct session *user_session =3D current_session();
= =C2=A0 bool is_err_action_default = =3D false;
@@ -1881,13 +1880,15 @@ xferOptimization(Parse * pPars= e, /* Parser context */
= =C2=A0
=C2=A0 /*
=C2=A0 * Xfer optimization is unab= le to correctly insert data
- * in case there's a conflict action other than
- * explicit *_ABORT. This is the reason we = want to only
+ * in case = there's a conflict action other than *_ABORT,
+ * *_FAIL or *_IGNORE. This is the reason we want= to only
=C2=A0 * run it = if the destination table is initially empty.
=C2=A0 * That block generates code to make that determi= nation.
=C2=A0 */
- if (!(onError =3D=3D ON_CONFLIC= T_ACTION_ABORT &&
- =C2=A0 =C2=A0 !is_err_action_default)) {
+ if (!(onError =3D=3D ON_CONFLICT_ACTION_ABORT ||
+ =C2=A0 =C2=A0 onError =3D=3D ON= _CONFLICT_ACTION_FAIL ||
+ =C2=A0 =C2=A0 onError =3D=3D ON_CONFLICT_ACTION_IGNORE) ||
+ =C2=A0 =C2=A0 is_err_action_default) {<= /div>
=C2=A0 addr1 =3D sqlite3V= dbeAddOp2(v, OP_Rewind, iDest, 0);
=C2=A0 VdbeCoverage(v);
=C2=A0 emptyDestTest =3D sqlite3VdbeAddOp0(v, OP_Goto);
= @@ -1905,15 +1906,23 @@ xferOptimization(Parse * pParse, /* Parser context */
=C2=A0#endif
= =C2=A0
=C2=A0 sqlite3VdbeA= ddOp2(v, OP_IdxInsert, iDest, regData);
- sqlite3VdbeChangeP5(v, OPFLAG_NCHANGE);
+ switch (onError) {
+ case ON_CONFLICT_ACTION_IGNORE:
+ sqlite3VdbeChangeP5(v, OPFLAG_OE_IGNO= RE);
+ break;
+= case ON_CONFLICT_ACTION_FAIL:
=
+ sqlite3VdbeChangeP5(v, OPFLA= G_OE_FAIL);
+ break;
+ default:
+ sqlite3VdbeChangeP5(v, OPFLAG_NCHANGE);
+ break;
+ }
=C2=A0 sqlite3VdbeAddOp2(v, OP_Next, iSrc, addr1 + 1);
= =C2=A0 VdbeCoverage(v);
= =C2=A0 sqlite3VdbeJumpHere(v, addr1)= ;
=C2=A0 sqlite3VdbeAddOp2= (v, OP_Close, iSrc, 0);
=C2=A0 sqlite3VdbeAddOp2(v, OP_Close, iDest, 0);
=C2=A0
-= if (emptySrcTest)
- sqlite3VdbeJumpHere(v, emptySrcTest);
=C2=A0 sqlite3ReleaseTempReg= (pParse, regTupleid);
=C2=A0 sqlite3ReleaseTempReg(pParse, regData);
=C2=A0 if (emptyDestTest) {
diff --git a/src/box= /sql/vdbe.c b/src/box/sql/vdbe.c
index aa7f250..ce01039 100644
--- a/src/box/sql/vdbe.c
+++ b/src/box/sql/vdbe.c
@@ -79,10 +79,10 @@ int sql_search_count =3D 0;
=C2=A0
=C2=A0#ifdef SQLITE_TEST
=C2=A0/*
- * The following = global variable is incremented whenever the
- * transfer optimiza= tion is used.=C2=A0 This is used for testing
- * purposes only - = to make sure the transfer optimization really
- * is happening wh= en it is supposed to.
+ * The following global variable is increm= ented in OP_RowData
+ * whenever the xfer optimization is used. T= his is used on
+ * testing purposes only - to make sure the trans= fer optimization
+ * really is happening when it is supposed to.<= /div>
=C2=A0 */
=C2=A0int sql_xfer_count =3D 0;
=C2= =A0#endif
@@ -4008,9 +4008,13 @@ case OP_RowData: {
=C2= =A0 BtCursor *pCrsr;
=C2= =A0 u32 n;
=C2=A0
+/*
+ * Flag P5 is cleared after the first insertion using xfer=
+ * optimization.
+ */
=C2=A0#ifdef SQLITE_T= EST
=C2=A0 if ((pOp->p5= & OPFLAG_XFER_OPT) !=3D 0) {
- pOp->p5 =3D 0;
+ <= /span>pOp->p5 &=3D ~OPFLAG_XFER_OPT;
=C2=A0 sql_xfer_count++;
=C2=A0 }
=C2=A0#endif
diff --git a/te= st/sql-tap/gh-3307-xfer-optimization-issue.test.lua b/test/sql-tap/gh-3307-= xfer-optimization-issue.test.lua
index 05d30c6..b99de2b 100755
--- a/test/sql-tap/gh-3307-xfer-optimization-issue.test.lua
+++ b/test/sql-tap/gh-3307-xfer-optimization-issue.test.lua
@@= -2,24 +2,28 @@
=C2=A0test =3D require("sqltester")
=C2=A0test:plan(46)
=C2=A0
-local bfr, aftr
-
-local function do_xfer_test(test_number, return_code)
- test_name =3D string.forma= t("xfer-optimization-1.%d", test_number)
- test:do_test(
- test_name,
- = function()
- ret= urn {aftr - bfr}
- end, {=
- -- <test_name><= /div>
- return_code
= - -- <test_name>
-= })
+local function do_xf= er_test(test, test_func, test_name, func, exp, opts)
+ local opts =3D opts or {}
+ local exp_xfer_count =3D opts.exp_xfer_count=
+ local before =3D box.sq= l.debug().sql_xfer_count
+ local ok, result =3D pcall(test_func, test, test_name, func, exp)
+ local after =3D box.sql.debug().= sql_xfer_count
+ if exp_xf= er_count ~=3D nil then
+ = ok =3D ok and test:is(after - before, exp_xfer_count,
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 test_name .. '-xfer-count')
+= end
+ return ok
=C2=A0end
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
+test.do_execsql_xfer_t= est =3D function(test, test_name, func, exp, opts)
+ return do_xfer_test(test, test.do_execsql_test= , test_name, func, exp, opts)
+end
+
+test.do= _catchsql_xfer_test =3D function(test, test_name, func, exp, opts)
+ return do_xfer_test(test, test.d= o_catchsql_test, test_name, func, exp, opts)
+end
=C2= =A0
-test:do_catchsql_test(
+test:do_catchsql_xfer_test= (
=C2=A0 "xfer-optimi= zation-1.1",
=C2=A0 [= [
=C2=A0 CREATE TABLE t1(= a INTEGER PRIMARY KEY, b INTEGER UNIQUE);
@@ -30,10 +34,10 @@ tes= t:do_catchsql_test(
=C2=A0 -- <xfer-optimization-1.1>
=C2=A0 0
=C2=A0 -- <xfer-optimization-1.1>
+ }, {
+ exp_xfer= _count =3D 1
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
= -
=C2=A0test:do_execsql_test(
=C2=A0 "xfer-optimization-1.2",
=C2=A0 [[
@@ -44,12 +48,8 @@ test:d= o_execsql_test(
=C2=A0 --= <xfer-optimization-1.2>
=C2=A0 })
=C2=A0
-do_xfer_test(3, 1)
-
-bfr =3D box.sql.debug().sql_xfer_count
-
-tes= t:do_catchsql_test(
- &quo= t;xfer-optimization-1.4",
+test:do_catchsql_xfer_test(
=
+ "xfer-optimization-1.3&q= uot;,
=C2=A0 [[
= =C2=A0 DROP TABLE t1;
=C2= =A0 DROP TABLE t2;
@@ -60= ,15 +60,15 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3);
= =C2=A0 INSERT INTO t2 SELECT * FROM= t1;
=C2=A0 ]], {
- -- <xfer-optimization-1.4>= ;
+ -- <xfer-optimizat= ion-1.3>
=C2=A0 0
- -- <xfer-optimization-1.= 4>
+ -- <xfer-optim= ization-1.3>
+ }, {
+ exp_xfer_count =3D 1
=
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
-
=C2=A0= test:do_execsql_test(
- &q= uot;xfer-optimization-1.5",
+ "xfer-optimization-1.4",
=C2=A0 [[
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
@@ -77,12 +77,8 @@ test:do_execsql_test(
=C2=A0 -- <xfer-optimization-1.= 5>
=C2=A0 })
= =C2=A0
-do_xfer_test(6, 1)
-
-bfr =3D box.sql= .debug().sql_xfer_count
-
-test:do_catchsql_test(
=
- "xfer-optimization-1.7&q= uot;,
+test:do_catchsql_xfer_test(
+ "xfer-optimization-1.5",
=C2=A0 [[
=C2=A0 DROP TABLE t1;
=C2=A0 DROP TABLE t2;
@@ -91,29 +87,25 @@ test:do_cat= chsql_test(
=C2=A0 CREATE= TABLE t2(a INTEGER PRIMARY KEY, b INTEGER);
=C2=A0 INSERT INTO t2 SELECT * FROM t1;
=C2= =A0 ]], {
- -- <xfer-optimization-1.7>
+ -- <xfer-optimization-1.5>
<= div>=C2=A0 1, "table T2 has 2 = columns but 3 values were supplied"
- -- <xfer-optimization-1.7>
+ -- <xfer-optimization-1.5>
+<= span style=3D"white-space:pre"> }, {
+ exp_xfer_count =3D 0
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debu= g().sql_xfer_count
-
=C2=A0test:do_execsql_test(
<= div>- "xfer-optimization-1.8&qu= ot;,
+ "xfer-optimiza= tion-1.6",
=C2=A0 [[<= /div>
=C2=A0 SELECT * FROM t2;<= /div>
=C2=A0 ]], {
- -- <xfer-optimization-1.8>
+ -- <xfer-optimization-1= .6>
=C2=A0
- -- <xfer-optimization-1.8>
+ -- <xfer-optimization-1.6>
=C2=A0 })
=C2=A0
-do_xfer_test(9, 0)<= /div>
-
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do_catchsql_test(
- "xfer-optimization-1.10",
+test:do_catchsql_xf= er_test(
+ "xfer-opti= mization-1.7",
=C2=A0 [[
=C2=A0 DROP TABLE t1;=
=C2=A0 DROP TABLE t2;
@@ -122,29 +114,25 @@ test:do_catchsql_test(
=C2=A0 CREATE TABLE t2(a INTEGER PRIMARY KEY, b= INTEGER);
=C2=A0 INSERT = INTO t2 SELECT * FROM t1;
=C2=A0 = ]], {
- -- <xfe= r-optimization-1.10>
+ -- <xfer-optimization-1.7>
=C2=A0 0
- -- &l= t;xfer-optimization-1.10>
+ <= /span>-- <xfer-optimization-1.7>
+ }, {
+ exp= _xfer_count =3D 1
=C2=A0 }= )
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
=
-
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.11",
+ "xfer-optimization-1.8",
=C2=A0 [[
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.11>
+ -- <xfer-optimization-1.8>
=C2= =A0 1, 1, 2, 2, 3, 3
- -- <xfer-optimization-1.11>
+ -- <xfer-optimization-1= .8>
=C2=A0 })
=C2=A0
-do_xfer_test(12, 1);
-
-bfr =3D box.= sql.debug().sql_xfer_count
-
-test:do_catchsql_test(
- "xfer-optimization-1.= 13",
+test:do_catchsql_xfer_test(
+ "xfer-optimization-1.9",
=C2= =A0 [[
=C2=A0 DROP TABLE t1;
=C2=A0 DROP TABLE t2;
@@ -153,29 +141,25 @@ t= est:do_catchsql_test(
=C2=A0 CREATE TABLE t2(b INTEGER, a INTEGER PRIMARY KEY);
=C2=A0 INSERT INTO t2 SELECT * FROM t1;
=
=C2=A0 ]], {
- -- <xfer-optimization-1.13>
+ -- <xfer-optimization-1.9&g= t;
=C2=A0 1, "Duplic= ate key exists in unique index 'sqlite_autoindex_T2_1' in space = 9;T2'"
- -- <= xfer-optimization-1.13>
+ -- <xfer-optimization-1.9>
+ }, {
+ exp_x= fer_count =3D 0
=C2=A0 })<= /div>
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
-
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.14",
+ "xfer-optimization-1.10",
=C2=A0 [[
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.14>
+ -- <xfer-optimization-1.10>
=C2= =A0
- -- <xfer-optimiz= ation-1.14>
+ -- <x= fer-optimization-1.10>
=C2=A0 = })
=C2=A0
-do_xfer_test(15, 0)
-
=
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do= _catchsql_test(
- "xf= er-optimization-1.16",
+test:do_catchsql_xfer_test(
+ "xfer-optimization-1.11&quo= t;,
=C2=A0 [[
= =C2=A0 DROP TABLE t1;
=C2= =A0 DROP TABLE t2;
@@ -18= 4,25 +168,23 @@ test:do_catchsql_test(
=C2=A0 CREATE TABLE t2(b INTEGER PRIMARY KEY, a INTEGER);
=C2=A0 INSERT INTO t2 SELECT= * FROM t1;
=C2=A0 ]], {
- -- <xfer-optimization= -1.16>
+ -- <xfer-o= ptimization-1.11>
=C2=A0 0
- -- <xfer-optimi= zation-1.16>
+ -- <= xfer-optimization-1.11>
+ }, {
+ exp_xfer_count = =3D 1
=C2=A0 })
= =C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
-
=
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.17",
+ "xfer-optimization-1.12",
=C2=A0 [[
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.17>
+ -- <xfer-optimization-1.12>
=C2=A0 1, 1, 2, 2, 3, 2
- -- <xfer-optimization-1.17>
+ -- <xfer-optimization-1.12>
=C2=A0 })
=C2=A0
-do_xfer_test(18, 1)
-
=C2=A0-- The following te= sts are supposed to test if xfer-optimization is actually
=C2=A0-= - used in the given cases (if the conflict actually occurs):
=C2= =A0-- 1.0) insert w/o explicit conf= l. action & w/o index replace action
@@ -221,10 +203,8 @@ do_= xfer_test(18, 1)
=C2=A0-- 1.0) insert w/o explicit confl. action = & w/o index replace action
=C2=A0----------------------------= --------------------------------------------------
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do_c= atchsql_test(
- "xfer= -optimization-1.19",
+test:do_catchsql_xfer_test(
= + "xfer-optimization-1.13"= ,
=C2=A0 [[
=C2= =A0 DROP TABLE t1;
=C2=A0= DROP TABLE t2;
@@ -236,3= 5 +216,31 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT INTO t2 SELECT * FROM t1;
= =C2=A0 ]], {
- -- <xfer-optimization-1.19>
+= -- <xfer-optimization-1.13><= /div>
=C2=A0 1, "Duplicate= key exists in unique index 'sqlite_autoindex_T2_1' in space 'T= 2'"
- -- <xfe= r-optimization-1.19>
+ -- <xfer-optimization-1.13>
+ }, {
+ exp_xfe= r_count =3D 0
=C2=A0 })
=C2=A0
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.20",
= + "xfer-optimization-1.14"= ,
=C2=A0 [[
=C2= =A0 INSERT INTO t2 VALUES (10, 10)= ;
=C2=A0 COMMIT;
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.20>
= + -- <xfer-optimization-1.14>=
=C2=A0 2, 2, 3, 4, 4, 4,= 10, 10
- -- <xfer-opt= imization-1.20>
+ -- &= lt;xfer-optimization-1.14>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfer= _count
-
-do_xfer_test(21, 0)
-
=C2= =A0-- 1.1) insert w/o explicit confl. action & w/
=C2=A0--=C2= =A0 =C2=A0 =C2=A0 index replace action & empty dest_table
=C2= =A0------------------------------------------------------------------------= ------
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do_catchsql_test(
- "xfer-optimization-1.22",
+test:do= _catchsql_xfer_test(
+ &qu= ot;xfer-optimization-1.15",
=C2=A0 [[
=C2=A0 D= ROP TABLE t1;
=C2=A0 DROP= TABLE t2;
@@ -276,45 +252,41 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t3 VALUES (1);=
=C2=A0 INSERT INTO t2 S= ELECT * FROM t1;
=C2=A0 ]]= , {
- -- <xfer-optimiz= ation-1.22>
+ -- <x= fer-optimization-1.15>
=C2=A0 = 0
- -- <xfer-o= ptimization-1.22>
+ --= <xfer-optimization-1.15>
+= }, {
+ exp_xfer_c= ount =3D 1
=C2=A0 })
=
=C2=A0
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.23",
+ "xfer-optimization-1.16",
=C2=A0 [[
=C2=A0<= span style=3D"white-space:pre"> INSERT INTO t2 VALUES (10, 10);
=C2=A0 COMMIT;
= =C2=A0 SELECT * FROM t2;
= =C2=A0 ]], {
- -- <xfer-optimization-1.23>
+= -- <xfer-optimization-1.16><= /div>
=C2=A0 1, 1, 3, 3, 5, 5, = 10, 10
- -- <xfer-opti= mization-1.23>
+ -- &l= t;xfer-optimization-1.16>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_= count
-
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.24",
+ "xfer-optimization-1.17&qu= ot;,
=C2=A0 [[
= =C2=A0 SELECT * FROM t3;
= =C2=A0 ]], {
- -- <xfer-optimization-1.24>
+= -- <xfer-optimization-1.17><= /div>
=C2=A0 1
- -- <xfer-optimization-1.24>
=
+ -- <xfer-optimization-1.1= 7>
=C2=A0 })
= =C2=A0
-do_xfer_test(25, 1)
-
=C2=A0-- 1.2) i= nsert w/o explicit confl. action & w/
=C2=A0-- index replace = action & non-empty dest_table
=C2=A0-------------------------= -----------------------------------------------------
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:d= o_catchsql_test(
- "x= fer-optimization-1.26",
+test:do_catchsql_xfer_test(
+ "xfer-optimization-1.18&qu= ot;,
=C2=A0 [[
= =C2=A0 DROP TABLE t1;
=C2= =A0 DROP TABLE t2;
@@ -32= 7,34 +299,30 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT INTO t2 SELECT * FROM t1;
=C2=A0 ]], {
- -- <xfer-optimization-1.26>
+ -- <xfer-optimization-1.18>= ;
=C2=A0 0
- -- <xfer-optimization-1.26>
+ -- <xfer-optimization-1= .18>
+ }, {
+= exp_xfer_count =3D 0
=C2= =A0 })
=C2=A0
= =C2=A0test:do_execsql_test(
- "xfer-optimization-1.27",
+ "xfer-optimization-1.19",
=C2=A0 [[
=C2=A0 INSERT INTO t2 VALUES (10, 10);
=C2=A0 COMMIT;
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.27>
+ -- <xfer-optimization-1.19>
=C2=A0 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 10, 10
<= div>- -- <xfer-optimization-1.27= >
+ -- <xfer-optimi= zation-1.19>
=C2=A0 })<= /div>
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
-
-do_xfer_test(28, 0)
-
=C2=A0-- 2) inser= t with abort
=C2=A0----------------------------------------------= --------------------------------
=C2=A0
-bfr =3D box.sq= l.debug().sql_xfer_count
-
-test:do_catchsql_test(
- "xfer-optimization-1.29= ",
+test:do_catchsql_xfer_test(
+ "xfer-optimization-1.20",
=C2=A0= [[
=C2=A0 DROP TABLE t1;
=C2=A0 DROP TABLE t2;
@@ -366,34 +334,30 @@ test:d= o_catchsql_test(
=C2=A0 = INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT OR ABORT INTO t2 SELECT * FROM t1;
=C2=A0 ]], {
- -- <xfer-optimization-1.29>
+ -- <xfer-optimization-1.20>
= =C2=A0 1, "Duplicate key exist= s in unique index 'sqlite_autoindex_T2_1' in space 'T2'&quo= t;
- -- <xfer-optimiza= tion-1.29>
+ -- <xf= er-optimization-1.20>
+ }, {
+ exp_xfer_count = =3D 1
=C2=A0 })
= =C2=A0
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.30",
+ "xfer-optimization-1.21",
<= div>=C2=A0 [[
=C2=A0 INSERT INTO t2 VALUES (10, 10);
=C2=A0 COMMIT;
=C2=A0<= span style=3D"white-space:pre"> SELECT * FROM t2;
=C2=A0<= span style=3D"white-space:pre"> ]], {
- -- <xfer-optimization-1.30>
+ -- <xfer-optimization-1.21>
=C2=A0 2, 2, 3, 4, 4, 4, 10, 10
- -- <xfer-optimization= -1.30>
+ -- <xfer-o= ptimization-1.21>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
-
-do_xfer_test(31, 1)
-
=C2=A0-- 3.0= ) insert with rollback (into empty table)
=C2=A0-----------------= -------------------------------------------------------------
=C2= =A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do_catchsql_test(
- "xfer-optimization-1.32",
+test:do_catchsql_xfer_test(=
+ "xfer-optimization= -1.22",
=C2=A0 [[
=C2=A0 DROP TABLE t1;
<= div>=C2=A0 DROP TABLE t2;
@@ -403,34 +367,30 @@ test:do_catchsql_test(
=C2=A0 BEGIN;
=C2=A0 INSERT OR ROLLBACK INTO t2 SELECT * FROM t1;
= =C2=A0 ]], {
- -- <xfer-optimization-1.32>
+= -- <xfer-optimization-1.22><= /div>
=C2=A0 0
- -- <xfer-optimization-1.32>
=
+ -- <xfer-optimization-1.2= 2>
+ }, {
+ exp_xfer_count =3D 1
=C2= =A0 })
=C2=A0
= =C2=A0test:do_execsql_test(
- "xfer-optimization-1.33",
+ "xfer-optimization-1.23",
=C2=A0 [[
=C2=A0 INSERT INTO t2 VALUES (10, 10);
=C2=A0 COMMIT;
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.33>
+ -- <xfer-optimization-1.23>
=C2=A0 1, 1, 3, 3, 5, 5, 10, 10
- -- <xfer-optimization-1.33>
+ -- <xfer-optimization-1.23&= gt;
=C2=A0 })
= =C2=A0
-aftr =3D box.sql.debug().sql_xfer_count
-
=
-do_xfer_test(34, 1)
-
=C2=A0-- 3.1) insert with r= ollback (into non-empty table)
=C2=A0----------------------------= --------------------------------------------------
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do_c= atchsql_test(
- "xfer= -optimization-1.35",
+test:do_catchsql_xfer_test(
= + "xfer-optimization-1.24"= ,
=C2=A0 [[
=C2= =A0 DROP TABLE t1;
=C2=A0= DROP TABLE t2;
@@ -442,3= 2 +402,28 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT OR ROLLBACK INTO t2 SELECT * FROM t1= ;
=C2=A0 ]], {
-= -- <xfer-optimization-1.35><= /div>
+ -- <xfer-optimizatio= n-1.24>
=C2=A0 1, &quo= t;UNIQUE constraint failed: T2.A"
- -- <xfer-optimization-1.35>
+ -- <xfer-optimization-1.24>
+ }, {
+ exp_xfer_count =3D 0
=C2=A0 })
=C2=A0
=C2=A0test:do_execsql_te= st(
- "xfer-optimizat= ion-1.36",
+ "xf= er-optimization-1.25",
=C2=A0 [[
=C2=A0 SELECT= * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization= -1.36>
+ -- <xfer-o= ptimization-1.25>
=C2=A0 2, 2, 3, 4
- -- <xf= er-optimization-1.36>
+ -- <xfer-optimization-1.25>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sq= l_xfer_count
-
-do_xfer_test(37, 0)
-
=C2=A0-- 4) insert with replace
=C2=A0------------------------= ------------------------------------------------------
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:= do_catchsql_test(
- "= xfer-optimization-1.38",
+test:do_catchsql_xfer_test(
<= div>+ "xfer-optimization-1.26&q= uot;,
=C2=A0 [[
= =C2=A0 DROP TABLE t1;
=C2= =A0 DROP TABLE t2;
@@ -47= 9,34 +435,30 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT OR REPLACE INTO t2 SELECT * FROM = t1;
=C2=A0 ]], {
- -- <xfer-optimization-1.38>= ;
+ -- <xfer-optimizat= ion-1.26>
=C2=A0 0
- -- <xfer-optimization-1= .38>
+ -- <xfer-opt= imization-1.26>
+ }, {<= /div>
+ exp_xfer_count =3D 0
=C2=A0 })
=C2=A0
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.39",
+ "xfer-optimization-1.27",
=C2= =A0 [[
=C2=A0 INSERT INTO t2 VALUES (10, 10);
= =C2=A0 COMMIT;
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.39>
+ -- <xfer-optimization-1.27>
= =C2=A0 1, 1, 2, 2, 3, 3, 4, 4, 5, 5= , 10, 10
- -- <xfer-op= timization-1.39>
+ -- = <xfer-optimization-1.27>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfe= r_count
-
-do_xfer_test(40, 0)
-
= =C2=A0-- 5) insert with fail
=C2=A0------------------------------= ------------------------------------------------
=C2=A0
-bfr =3D box.sql.debug().sql_xfer_count
-
-test:do_cat= chsql_test(
- "xfer-o= ptimization-1.41",
+test:do_catchsql_xfer_test(
+<= span style=3D"white-space:pre"> "xfer-optimization-1.28",<= /div>
=C2=A0 [[
=C2=A0= DROP TABLE t1;
=C2=A0 DROP TABLE t2;
@@ -518,34 += 470,30 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT OR FAIL INTO t2 SELECT * FROM t1;
=C2=A0 ]], {
- -- <xfer-optimization-1.41>
=
+ -- <xfer-optimization-1.2= 8>
=C2=A0 1, "Dup= licate key exists in unique index 'sqlite_autoindex_T2_1' in space = 'T2'"
- -- &= lt;xfer-optimization-1.41>
+ = -- <xfer-optimization-1.28>
+ }, {
+ e= xp_xfer_count =3D 1
=C2=A0 })
=C2=A0
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.42",
+ "xfer-optimization-1.29= ",
=C2=A0 [[
=C2=A0 INSERT INTO t2 VALUES (10= , 10);
=C2=A0 COMMIT;
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.42>
=
+ -- <xfer-optimization-1.2= 9>
=C2=A0 1, 1, 2, 2, = 3, 4, 4, 4, 10, 10
- -- &= lt;xfer-optimization-1.42>
+ = -- <xfer-optimization-1.29>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug= ().sql_xfer_count
-
-do_xfer_test(43, 0)
-
=C2=A0-- 6) insert with ignore
=C2=A0--------------------= ----------------------------------------------------------
=C2=A0=
-bfr =3D box.sql.debug().sql_xfer_count
-
-t= est:do_catchsql_test(
- &q= uot;xfer-optimization-1.44",
+test:do_catchsql_xfer_test(
+ "xfer-optimization-1.= 30",
=C2=A0 [[
<= div>=C2=A0 DROP TABLE t1;
=C2=A0 DROP TABLE t2;
@@= -557,25 +505,23 @@ test:do_catchsql_test(
=C2=A0 INSERT INTO t2 VALUES (4, 4);
=C2=A0 INSERT OR IGNORE INTO t2 SELECT * FR= OM t1;
=C2=A0 ]], {
<= div>- -- <xfer-optimization-1.44= >
+ -- <xfer-optimi= zation-1.30>
=C2=A0 0<= /div>
- -- <xfer-optimizatio= n-1.44>
+ -- <xfer-= optimization-1.30>
+ },= {
+ exp_xfer_count =3D 1=
=C2=A0 })
=C2= =A0
=C2=A0test:do_execsql_test(
- "xfer-optimization-1.45",
+ "xfer-optimization-1.31",
=C2=A0 [[
=C2=A0 INSERT INTO t2 VALUES (10, 10);
= =C2=A0 COMMIT;
=C2=A0 SELECT * FROM t2;
=C2=A0 ]], {
- -- <xfer-optimization-1.45>
+ -- <xfer-optimization-1.31>
= =C2=A0 1, 1, 2, 2, 3, 4, 4, 4, 5, 5= , 10, 10
- -- <xfer-op= timization-1.45>
+ -- = <xfer-optimization-1.31>
=C2=A0 })
=C2=A0
-aftr =3D box.sql.debug().sql_xfe= r_count
-
-do_xfer_test(46, 0)
-
= =C2=A0test:finish_test()
diff --git a/test/sql-tap/with2.test.lua= b/test/sql-tap/with2.test.lua
index fbd1f4e..bd0187f 100755
--- a/test/sql-tap/with2.test.lua
+++ b/test/sql-tap/with2.= test.lua
@@ -1,6 +1,6 @@
=C2=A0#!/usr/bin/env tarantool=
=C2=A0test =3D require("sqltester")
-test:pl= an(44)
+test:plan(59)
=C2=A0
=C2=A0--!./tclte= strunner.lua
=C2=A0-- 2014 January 11
@@ -382,43 +382,1= 36 @@ genstmt(255), {
=C2=A0--=C2=A0 =C2=A0 =C2=A0-- </4.7>=
=C2=A0-- })
=C2=A0
-------------------------= -----------------------------------------------------
--- Check t= hat adding a WITH clause to an INSERT disables the xfer=C2=A0
+--= ---------------------------------------------------------------
+= -- Check that adding a WITH clause to an INSERT disables the xfer
=C2=A0-- optimization.
---
--- Tarantool: `sqlite3_xfe= ropt_count` is not exported
---=C2=A0 =C2=A0Need to understand if= this optimization works at all and if we really need it.
---=C2= =A0 =C2=A0Commented so far.
--- function do_xfer_test(tn, bXfer, = sql, res)
---=C2=A0 =C2=A0 =C2=A0res =3D res or ""
---=C2=A0 =C2=A0 =C2=A0sqlite3_xferopt_count =3D 0
---=C2= =A0 =C2=A0 =C2=A0X(267, "X!cmd", [=3D[["uplevel",[[&quo= t;list","do_test",["tn"],["\n=C2=A0 =C2=A0 se= t dres [db eval {",["sql"],"}]\n=C2=A0 =C2=A0 list [set= ::sqlite3_xferopt_count] [set dres]\n=C2=A0 "],[["list",[&q= uot;bXfer"],["res"]]]]]]]=3D])
--- end
= =C2=A0
--- test:do_execsql_test(
---=C2=A0 =C2=A0 =C2= =A05.1,
---=C2=A0 =C2=A0 =C2=A0[[
---=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0DROP TABLE IF EXISTS t1;
---=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0DROP TABLE IF EXISTS t2;
---=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0CREATE TABLE t1(a PRIMARY KEY, b);
---=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0CREATE TABLE t2(a PRIMARY KEY, b);
---=C2=A0 = =C2=A0 =C2=A0]])
-
--- do_xfer_test(5.2, 1, " INSE= RT INTO t1 SELECT * FROM t2 ")
--- do_xfer_test(5.3, 0, &quo= t; INSERT INTO t1 SELECT a, b FROM t2 ")
--- do_xfer_test(5.= 4, 0, " INSERT INTO t1 SELECT b, a FROM t2 ")
--- do_xf= er_test(5.5, 0, [[=C2=A0
---=C2=A0 =C2=A0WITH x AS (SELECT a, b F= ROM t2) INSERT INTO t1 SELECT * FROM x=C2=A0
--- ]])
--= - do_xfer_test(5.6, 0, [[=C2=A0
---=C2=A0 =C2=A0WITH x AS (SELECT= a, b FROM t2) INSERT INTO t1 SELECT * FROM t2=C2=A0
--- ]])
--- do_xfer_test(5.7, 0, [[=C2=A0
---=C2=A0 INSERT INTO t1 = WITH x AS ( SELECT * FROM t2 ) SELECT * FROM x
--- ]])
= --- do_xfer_test(5.8, 0, [[=C2=A0
---=C2=A0 INSERT INTO t1 WITH x= (a,b) AS ( SELECT * FROM t2 ) SELECT * FROM x
--- ]])
+= local function do_xfer_test(test, test_func, test_name, func, exp, opts)
+ local opts =3D opts or {}<= /div>
+ local exp_xfer_count =3D= opts.exp_xfer_count
+ loc= al before =3D box.sql.debug().sql_xfer_count
+ local ok, result =3D pcall(test_func, test, test_name,= func, exp)
+ local after = =3D box.sql.debug().sql_xfer_count
+ if exp_xfer_count ~=3D nil then
+ ok =3D ok and test:is(after - before, exp_xfer_count,
+ =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 test_name .. '-xfer-count= ')
+ end
+ return ok
+end
+<= /div>
+test.do_execsql_xfer_test =3D function(test, test_name, func, ex= p, opts)
+ return do_xfer_= test(test, test.do_execsql_test, test_name, func, exp, opts)
+end=
+
+test.do_catchsql_xfer_test =3D function(test, test_= name, func, exp, opts)
+ r= eturn do_xfer_test(test, test.do_catchsql_test, test_name, func, exp, opts)=
+end
+
+test:do_execsql_test(
+ 5.1,
+ [[
+ DR= OP TABLE IF EXISTS t1;
+ = DROP TABLE IF EXISTS t2;
+ CREATE TABLE t1(a PRIMARY KEY, b);
+ CREATE TABLE t2(a PRIMARY KEY, b);
+ INSERT INTO t2 VALUES (1, 1), (2, 2);
+= ]], {
+ -- <5.1>
+
+ -- <5.1>
+ })
+
+test:do_execsql_xfer_test(
+ 5.2,
+ [[
+= INSERT INTO t1 SELECT * FROM t2;
+ DELETE FROM t1;
+ ]], {
+ -- <= ;5.2>
+
+ --= <5.2>
+ },=C2=A0 {<= /div>
+ exp_xfer_count =3D 1
+ })
+
+= test:do_execsql_xfer_test(
+ 5.3,
+ [[
+ INSERT INTO t1 SELECT a, b FROM t2;<= /div>
+ DELETE FROM t1;
+ ]], {
+ -- <5.3>
+
+ -- <5.3>
+ },=C2=A0 {
+ exp_xfer_count =3D 0
+ = })
+
+test:do_execsql_xfer_test(
+ 5.4,
+ [[
+ INS= ERT INTO t1 SELECT b, a FROM t2;
+ DELETE FROM t1;
+ ]], {
+ -- <5.4><= /div>
+
+ -- <5.4&= gt;
+ },=C2=A0 {
+ exp_xfer_count =3D 0
+= })
+
+test:do_e= xecsql_xfer_test(
+ 5.5,
+ [[
+ WITH x AS (SELECT a, b FROM t2) INSERT INTO t= 1 SELECT * FROM x;
+ DELE= TE FROM t1;
+ ]], {
<= div>+ -- <5.5>
+
+ -- <5.5>
= + },=C2=A0 {
+ exp_xfer_count =3D 0
+ })
+
+test:do_execsql_xfer= _test(
+ 5.6,
+<= span style=3D"white-space:pre"> [[
+ WITH x AS (SELECT a, b FROM t2) INSERT INTO t1 SELECT * = FROM t2;
+ DELETE FROM t1= ;
+ ]], {
+ -- <5.6>
+
+<= span style=3D"white-space:pre"> -- <5.6>
+ },=C2=A0 {
+ exp_xfer_count =3D 0
+ })
+
+test:do_execsql_xfer_test(
<= div>+ 5.7,
+ [[
+ <= /span>INSERT INTO t1 WITH x AS (SELECT * FROM t2) SELECT * FROM x;
+ DELETE FROM t1;
+ ]], {
+ -- <5.7>
+
+ -- <5.7>
+ },=C2=A0 {
+ exp_xfer_count =3D 0
+ }= )
+
+test:do_execsql_xfer_test(
+ 5.8,
+ [[
+ INSERT INT= O t1 WITH x(a,b) AS (SELECT * FROM t2) SELECT * FROM x;
+ DELETE FROM t1;
+ ]], {
+ = -- <5.8>
+
+ -- <5.8>
+ },=C2=A0 {
+ exp_xfer_= count =3D 0
+ })
+
=C2=A0--------------------------------------------------------= ---------------------
=C2=A0-- Check that syntax (and other) erro= rs in statements with WITH clauses
=C2=A0-- attached to them do n= ot cause problems (e.g. memory leaks).
--0000000000005c49ca05724a2778--