From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 4DDF22925F for ; Tue, 27 Aug 2019 09:34:35 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5s-BdJTWvA5D for ; Tue, 27 Aug 2019 09:34:35 -0400 (EDT) Received: from smtpng3.m.smailru.net (smtpng3.m.smailru.net [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id C23A3289F8 for ; Tue, 27 Aug 2019 09:34:34 -0400 (EDT) From: Nikita Pettik Subject: [tarantool-patches] [PATCH 0/8] rfc: introduce dry-run execution of SQL queries Date: Tue, 27 Aug 2019 16:34:21 +0300 Message-Id: Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: v.shpilevoy@tarantool.org, kostja@tarantool.org, alexander.turenko@tarantool.org, Nikita Pettik Branch: https://github.com/tarantool/tarantool/tree/np/sql-dry-run-rfc Issue: https://github.com/tarantool/tarantool/issues/3292 In a nutshell dry-run is a way to get auxiliary (meta-) information of query execution without direct execution itself. For instance, every DQL query returns column's name and type alongside with tuples forming the resulting set. Meta-information can be extended with column's nullability, collation and other properties. This feature is required to develop SQL drivers. A bit of details. There are three forms of dumped msgpack structure (and corresponding IPROTO response format). First one relates to DQL (i.e. SELECT) queries: METADATA : [{name, type}, {name, type}, ...], DATA : [tuple, tuple ...] In case of dry-run, only metadata should be dumped. As a consequence, DATA section becomes optional and depends on port settings (see new 'meta_only' option in struct port_sql). To pass flag indicating dry-run execution, :execute() method has been allowed to accept 'options' argument. Options can form a map of named parameters ({'dry_run' = true}) or simply a list of unnamed ordered values. Options are encoded with IPROTO_OPTIONS key. The only available option now is 'dry_run'. To execute query in dry-run mode locally separate method of box has been added: box.dry_run("sql_string"). Current implementation does not modify box.execute() making it accept third optional argument (except for array of bindings) - this is to be discussed. Other open (or dubious) questions. 1. How to handle DML queries. Meanwhile getting meta-information of DDL queries is likely to be useless, calculating number of affected rows in case of DML query can be sensible. On the other hand, it can turn out to be quite non-trivial task. In its current state, box.dry_run("DML_Query") always returns 0 as row count. IMHO it can be done but as a follow-up 2. Should dry_run() except for parsing also bind variables? I guess no, since it results in byte-code (i.e. prepared statement) modification. On the other hand, it allows setting correct type for bindings (see sql_bind_type()). By default, all binding parameters have boolean type. 3. Interface for local call. In current implementation I've added box.dry_run() in addition to traditional box.execute() method. Instead, we can make box.execute() accept third argument as it is handled in net.box. Nikita Pettik (8): port: increase padding of struct port port: move struct port_sql to box/port.h sql: remove sql_prepare_v2() sql: refactor sql_prepare() and sqlPrepare() sql: move sql_prepare() declaration to box/execute.h refactoring: use sql_prepare() and sql_execute() in tx_process_sql() netbox: allow passing options to :execute() sql: introduce dry-run execution src/box/errcode.h | 1 + src/box/execute.c | 104 ++++++++++++++++++++++++++++++++++++----------- src/box/execute.h | 61 ++++++++++++++++++++------- src/box/iproto.cc | 22 +++++++++- src/box/lua/execute.c | 37 ++++++++++++++++- src/box/lua/net_box.c | 7 ++-- src/box/lua/net_box.lua | 3 -- src/box/port.h | 26 ++++++++++++ src/box/sql/analyze.c | 16 ++++---- src/box/sql/legacy.c | 3 +- src/box/sql/prepare.c | 38 +++-------------- src/box/sql/sqlInt.h | 16 -------- src/box/sql/vdbe.h | 2 +- src/box/sql/vdbeInt.h | 1 - src/box/sql/vdbeapi.c | 9 ++-- src/box/sql/vdbeaux.c | 5 +-- src/box/xrow.c | 19 +++++++-- src/box/xrow.h | 5 +++ src/lib/core/port.h | 3 +- test/box/misc.result | 2 + test/sql/iproto.result | 93 +++++++++++++++++++++++++++++++++++++++++- test/sql/iproto.test.lua | 19 +++++++++ 22 files changed, 371 insertions(+), 121 deletions(-) -- 2.15.1