Tarantool development patches archive
 help / color / mirror / Atom feed
From: Alexander Turenko <alexander.turenko@tarantool.org>
To: Nikita Tatunov <hollow653@gmail.com>
Cc: tarantool-patches@freelists.org
Subject: [tarantool-patches] Re: [PATCH] sql: xfer optimization issue
Date: Sun, 29 Jul 2018 04:12:51 +0300	[thread overview]
Message-ID: <20180729011251.eitp7cisv6jv5opj@tkn_work_nb> (raw)
In-Reply-To: <CAEi+_ap4Lc5QBPExitzSt_ZDMbF3ea=iqXFLGXDvwV+rY1MQ=Q@mail.gmail.com>

Hi!

Please consider my comments and questions below.

WBR, Alexander Turenko.

> +       /*
> +        * Xfer optimization is unable to correctly insert data
> +        * in case there's a conflict action other than
> +        * explicit *_ABORT. This is the reason we want to only
> +        * run it if the destination table is initially empty.
> +        * That block generates code to make that determination.
> +        */
> +       if (!(onError == ON_CONFLICT_ACTION_ABORT &&
> +           !is_err_action_default)) {

Do you mean that:

1. The optimization non-empty table case correctly works only with ABORT
   conflict action (default or explicit).
2. The default conflict action can be overwritten on per-column basis, so
   'default abort' can really be replace or other conflict action.

If so, your description doesn't give this information.

To more on that, can we check per-column conflict actions instead of check
whether the conflict action default or explicit? This would enable more cases
with non-empty tables for the optimization. And this would look less confusing,
IMHO.

I have one more question on that. It seems that SQLite has this optimization
working with ROLLBACK conflict action. We cannot doing so, because of some
problem? Did this problem described / trackerized somewhere? Or something
changes underhood and makes this impossible? Are we know exact reason or just
observing it does not work?

> +#ifdef SQLITE_TEST
> +/*
> + * The following global variable is incremented whenever the
> + * transfer optimization is used.  This is used for testing
> + * purposes only - to make sure the transfer optimization really
> + * is happening when it is supposed to.
> + */
> +int sql_xfer_count = 0;
> +#endif

I think it would be good to mention the opcode where the counter is
incremented. You can follow the style in which other counters are
described (they are mostly mention opcodes).

> -/* Opcode: RowData P1 P2 * * *
> +/* Opcode: RowData P1 P2 * * P5

We can increase the counter on per-operation basis instead of per-row by
adding the flag to OP_OpenWrite. It will save some CPU cycles :)

> +#ifdef SQLITE_TEST
> +       if ((pOp->p5 & OPFLAG_XFER_OPT) != 0) {
> +               pOp->p5 = 0;
> +               sql_xfer_count++;
> +       }
> +#endif

1. Not actual due to 2, but it would be better to use
   `pOp->p5 &= ~OPFLAG_XFER_OPT` to drop just that flag.
2. It is counter-intuitive, IMHO, to change operation flags during that
   operation. So, said above, vote to move it to OP_OpenWrite.

> +local bfr, aftr
> +

What do you plan to do with saved letters? :) Really, such abbreviations
just makes reading harder with no gains.

> +local function do_xfer_test(test_number, return_code)
> +       test_name = string.format("xfer-optimization-1.%d", test_number)
> +       test:do_test(
> +               test_name,
> +               function()
> +                       return {aftr - bfr}
> +               end, {
> +                       -- <test_name>
> +                       return_code
> +                       -- <test_name>
> +               })
> +end

That code can be written simpler (consider tap module documentation):

test:is(after - before, exp, test_name)

I suggest to create wrappers like so (I didn't test it):

local function do_xfer_test(test_func, test, test_name, func, exp, opts)
    local opts = opts or {}
    local exp_xfer_count = opts.exp_xfer_count
    local before = box.sql.debug().sql_xfer_count
    local ok = test_func(test, test_name, func, exp)
    local after = box.sql.debug().sql_xfer_count
    if exp_xfer_count ~= nil then
        ok = ok and test:is(after - before, exp_xfer_count, test_name ..
            '_xfer_count')
    end
    return ok
end

test.do_execsql_xfer_test = function(test, test_name, func, exp, opts)
    return do_xfer_test(test.do_execsql_test, test, test_name, func, exp, opts)
end

test.do_catchsql_xfer_test = function(test, test_name, func, exp, opts)
    return do_xfer_test(test.do_catchsql_test, test, test_name, func, exp, opts)
end

And use it like so:

test:do_catchsql_xfer_test(
    "xfer-optimization-1.1",
    [[
        CREATE TABLE t1(a INTEGER PRIMARY KEY, b INTEGER UNIQUE);
        INSERT INTO t1 VALUES (1, 1), (2, 2), (3, 3);
        CREATE TABLE t2(a INTEGER PRIMARY KEY, b INTEGER UNIQUE);
        INSERT INTO t2 SELECT * FROM t1;
    ]], {
        -- <xfer-optimization-1.1>
        0
        -- <xfer-optimization-1.1>
    }), {
        exp_xfer_count = 1
    }
)

By the way, you can revive xfer cases in test/sql-tap/with2.test.lua. Or drop
it if your new test includes all related cases from with2.

On Fri, Jul 20, 2018 at 07:58:48PM +0300, Nikita Tatunov wrote:
>    Ooops. Thank you! fixed it and pushed.
> 
>    пт, 20 июл. 2018 г. в 19:43, n.pettik <[1]korablev@tarantool.org>:
> 
>    LGTM.
> 
>    diff --git a/src/box/sql/insert.c b/src/box/sql/insert.c
>    index 3c3bf37..4f52fa5 100644
>    --- a/src/box/sql/insert.c
>    +++ b/src/box/sql/insert.c
>    @@ -1869,7 +1869,7 @@ xferOptimization(Parse * pParse, /* Parser
>    context */
>       * table (tab1) is initially empty.
>       */
> 
>    - /* The Vdbe we're building*/
>    + /* The Vdbe struct we're building. */
> 
>    You misunderstood me. What I mean is:
>    struct Vibe *v = …;
> 
>      Vdbe *v = sqlite3GetVdbe(pParse);
> 
> References
> 
>    1. mailto:korablev@tarantool.org

  reply	other threads:[~2018-07-29  1:12 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-18 15:32 [tarantool-patches] " N.Tatunov
2018-04-18 16:33 ` [tarantool-patches] " Hollow111
2018-04-19 11:22   ` n.pettik
2018-04-19 15:36     ` Hollow111
2018-04-20  1:02       ` n.pettik
2018-04-20 15:09         ` Hollow111
2018-04-20 16:09           ` n.pettik
2018-04-20 17:59             ` Hollow111
2018-04-23 23:40               ` n.pettik
2018-04-27 15:45                 ` Hollow111
2018-05-03 22:57                   ` n.pettik
2018-05-04 12:54                     ` Hollow111
2018-06-28 10:18                       ` Alexander Turenko
2018-07-09 15:50                         ` Alexander Turenko
2018-07-16 12:54                           ` Nikita Tatunov
2018-07-16 13:06                             ` n.pettik
2018-07-16 13:20                               ` Nikita Tatunov
2018-07-16 18:37                                 ` Nikita Tatunov
2018-07-16 19:12                                   ` n.pettik
2018-07-16 21:27                                     ` Nikita Tatunov
2018-07-18 15:13                                       ` n.pettik
2018-07-18 20:18                                         ` Nikita Tatunov
2018-07-19  0:20                                           ` n.pettik
2018-07-19 17:26                                             ` Nikita Tatunov
2018-07-20  3:20                                               ` n.pettik
2018-07-20 11:56                                                 ` Nikita Tatunov
2018-07-20 16:43                                                   ` n.pettik
2018-07-20 16:58                                                     ` Nikita Tatunov
2018-07-29  1:12                                                       ` Alexander Turenko [this message]
2018-07-29 11:23                                                         ` n.pettik
2018-07-29 15:16                                                           ` Alexander Turenko
2018-07-30 18:33                                                             ` Nikita Tatunov
2018-07-30 22:17                                                               ` Alexander Turenko
2018-07-31 11:48                                                         ` Nikita Tatunov
2018-07-31 13:29                                                           ` Alexander Turenko
2018-07-31 17:04                                                             ` Nikita Tatunov
2018-07-31 17:44                                                               ` Alexander Turenko
2018-08-21 16:43 ` Kirill Yukhin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180729011251.eitp7cisv6jv5opj@tkn_work_nb \
    --to=alexander.turenko@tarantool.org \
    --cc=hollow653@gmail.com \
    --cc=tarantool-patches@freelists.org \
    --subject='[tarantool-patches] Re: [PATCH] sql: xfer optimization issue' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox