From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp38.i.mail.ru (smtp38.i.mail.ru [94.100.177.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 0CB46469710 for ; Thu, 26 Nov 2020 10:45:17 +0300 (MSK) From: "Timur Safin" References: <0b1d01d6b9a4$aecaf500$0c60df00$@tarantool.org> <07e101d6c2ae$0607c980$12175c80$@tarantool.org> <20201125094456.GA23894@tarantool.org> In-Reply-To: <20201125094456.GA23894@tarantool.org> Date: Thu, 26 Nov 2020 10:45:12 +0300 Message-ID: <097e01d6c3c8$12ca6d60$385f4820$@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Content-Language: ru Subject: Re: [Tarantool-discussions] RFC - distributed SQL, step #1 - AST List-Id: Tarantool development process List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: 'Nikita Pettik' Cc: m.semkin@corp.mail.ru, mons@tarantool.org, tarantool-discussions@dev.tarantool.org, 'Alexander Turenko' : From: Nikita Pettik : Subject: Re: [Tarantool-discussions] RFC - distributed SQL, step #1 - = AST :=20 : On 25 Nov 01:06, Timur Safin wrote: : > : > Exporting AST as cdata is easy to do, but in this case it will be : > inconvenient to manipulate AST nodes, before sending to cluster = nodes. : > So there is 2nd, more idiomatic approach suggested - to convert AST : > to nested Lua object/tables. :=20 : Hm, why do you need those tables? Serialization into msgpack can : be done inside SQL internals. I do understand that direct conversion to msgpack from ast, and creation of tables instead of msgpack, for manipulations in Lua would have the = same complexity and mostly would be done using the same walker code. The = problem I foresee - it would be hard to massage AST in Lua iff we would need to. And we would have to do it eventually, once we approach distributed SQL = router task which might need to modify original AST before sending to data = nodes for their local execution. Do you remember how Mike had to process AST = cdata for special aggregation functions processing, and modification of column = list? That's why I assumed that exposing AST nodes as Lua tables would = simplify such modification task, I guessed it would be more idiomatic for Lua. But I = might be wrong here... :=20 : > Which should simplify some massaging : > and provide natural way to serialization to msgpack. : > : > : > SYNOPSIS : > ~~~~~~~~ : > : > .. code-block:: lua : > : > local sql =3D require `sqlparser` : > : > -- parse, return ast, pass it back unmodified for execution : > : > local ast =3D sql.parse [[ select * from "table" where id > 10 = limit 10 : ]] :=20 : Should this be public API? Alternatively, we can hide it in = sql.internals. I consider sqlparser the internal API already (i.e. box.sql instead of=20 currently used temporary sqlparser). So having yet another internal api=20 would not make much sense. But it's all up to our decision.=20 :=20 : > assert(type(ast) =3D=3D 'cdata') : > local ok =3D sql.execute(ast) : > : > -- free allocated memory, like box.unprepare but for ast : > sql.unparse(ast) :=20 : I don't like unparse name. In fact even unprepare is a bad name, : it should be called sort of deallocate. I suggest sql.release_ast()/ : sql.free_ast() naming. I'm ok with unprepare :) [we used this name in the InterSystems for = similar=20 Contexts] and thus ok with unparse. But seimilarly I'm ok with any = different=20 name - the major point here, there should be anything which frees AST = data structures kept elsewhere. :=20 : > -- raw access to cdata structures : > local cdata =3D ast.as_cdata() : > if cdata.ast_type =3D=3D ffi.C.AST_TYPE_SELECT : > handle_select_stmt(cdata.select) : > end : > : > -- Lua access to structurs as Lua tables : > local tree =3D ast.as_table() : > if tree.type =3D=3D AST_TYPE_SELECT : > handle_columns_list(tree.select.columns) : > handle_where_clause(tree.select.where) : > limit =3D tree.select.limit :=20 : What's the purpose of these handles? Sorry for the confusion created. Those **handle_anything** assumed=20 to represent any user-defined function which manipulate with cdata=20 exported. i.e. I should put it the way: user_handle_column_list(tree.select.columns) user_handle_where_clause(tree.select.where) and here we define only tree subobjects exposed to cdata, and don't care how and where user defined their functions traversing/manipulating=20 AST. :=20 : > end : > -- massaging with tree data : > : > -- serialization : > local msgpack =3D require 'msgpack' : > local to_wire =3D msgpack.encode(tree) : > : > -- networking magics ... : > -- ... deserialization : > local table =3D msgpack.decode(from_wire) : > ast.set_tree(tree) : > sql.execute(ast) : > : > : > Regards, : > Timur : > : > P.S. : >