From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 5E7A56ECC0; Mon, 13 Dec 2021 10:42:44 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 5E7A56ECC0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1639381364; bh=PpxnfNfGOqf5ivzqCbokp4TK8Z8q3jimPcpxCZQ+slc=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=L/ky7TX70vdQ11fpNmdmapJVtntZf7irrYxrS3LaY23w/5RZXRJ3uZud1MNBbzF5l qj1rEcWOA+rF6cMgJmAM7D+J33oS+toypb74fqwkv0wMerhpw5wr1DJJvBbSkdSbR8 ny//JAMis/fmzeiIhiuP5sdA5TvufjKvfgNHBUFI= Received: from smtpng3.i.mail.ru (smtpng3.i.mail.ru [94.100.177.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 7D95F6ECC0 for ; Mon, 13 Dec 2021 10:42:42 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 7D95F6ECC0 Received: by smtpng3.m.smailru.net with esmtpa (envelope-from ) id 1mwfyr-0006U9-Q2; Mon, 13 Dec 2021 10:42:42 +0300 Date: Mon, 13 Dec 2021 10:42:39 +0300 To: Vladislav Shpilevoy Cc: tarantool-patches@dev.tarantool.org Message-ID: <20211213074239.GB41198@tarantool.org> References: <4ecfb3439688bef76c96270624410dee8822176f.1637244389.git.imeevma@gmail.com> <662f6b87-f085-ab10-53b8-d087d9598b19@tarantool.org> <20211125085529.GB56448@tarantool.org> <20211202083851.GB8207@tarantool.org> <9b75890d-32e4-aaa9-c377-01854278bc76@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <9b75890d-32e4-aaa9-c377-01854278bc76@tarantool.org> X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD9FFF274446F725B74506C72DDA7EBE5A636C5C70934101B1A182A05F53808504021FE3F6F5384BD6490863895236A8347F1BB49FB433CAB55AC6F5E0284F870B3 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE768D1C4AD116E0413EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006378D071468F2F69688EA1F7E6F0F101C6723150C8DA25C47586E58E00D9D99D84E1BDDB23E98D2D38BBCA57AF85F7723F2E1D86BBB0DA7A7A18282D06A13CD466ACC7F00164DA146DAFE8445B8C89999728AA50765F7900637D0FEED2715E18529389733CBF5DBD5E9C8A9BA7A39EFB766F5D81C698A659EA7CC7F00164DA146DA9985D098DBDEAEC8062BEEFFB5F8EA3EF6B57BC7E6449061A352F6E88A58FB86F5D81C698A659EA7E827F84554CEF5019E625A9149C048EE9ECD01F8117BC8BEE2021AF6380DFAD18AA50765F790063735872C767BF85DA227C277FBC8AE2E8B08F9A42B2210255C75ECD9A6C639B01B4E70A05D1297E1BBCB5012B2E24CD356 X-C1DE0DAB: 0D63561A33F958A5E3FE9B11E2EB57E2D36F573CDF1098A827F333A5D618B7F7D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA759D2A03B9C34326B3410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D340A58BF0C7EAC340F739B9F793997A9BA06D02DC8BD548C66581B3CD78A59623F563862F319E1E5D61D7E09C32AA3244C28FD549A47A7B6F313AF1AFC488AAE0E97FE24653F78E668729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojHOet/aB0CsiitrKbholR1A== X-Mailru-Sender: 689FA8AB762F7393C37E3C1AEC41BA5DA94198CE9B30A5D9A8C9B004F1D63AAE83D72C36FC87018B9F80AB2734326CD2FB559BB5D741EB96352A0ABBE4FDA4210A04DAD6CC59E3365FEEDEB644C299C0ED14614B50AE0675 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v1 2/2] sql: introduce syntax for MAP values X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mergen Imeev via Tarantool-patches Reply-To: Mergen Imeev Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Thank you for the review! My answers and diff below. On Thu, Dec 09, 2021 at 01:31:39AM +0100, Vladislav Shpilevoy wrote: > Thanks for the fixes! > > >>>>> diff --git a/src/box/sql/expr.c b/src/box/sql/expr.c > >>>>> index 74a98c550..789d8906c 100644 > >>>>> --- a/src/box/sql/expr.c > >>>>> +++ b/src/box/sql/expr.c > >>>>> @@ -3432,6 +3432,35 @@ expr_code_array(struct Parse *parser, struct Expr *expr, int reg) > >>>>> sqlVdbeAddOp3(vdbe, OP_Array, count, reg, values_reg); > >>>>> } > >>>>> > >>>>> +static void > >>>>> +expr_code_map(struct Parse *parser, struct Expr *expr, int reg) > >>>> > >>>> 1. I thought the policy was that we name functions, generating VDBE code, > >>>> using 'emit' suffix. For instance, `vdbe_emit_map()` or `sql_emit_map()`. > >>>> Don't know about prefix though. I see both vdbe_ and sql_ are used. > >>>> > >>> This is usually true, but this function is actually part of sqlExprCodeTarget(). > >>> I believe these functions were created to make sqlExprCodeTarget() more > >>> readable. All such functions are named sqlExprCode*(), code*() or > >>> expr_code _*(), for example: sqlExprCodeGetColumn(), codeReal(), > >>> expr_code_int(). > >>> > >>> Since all these functions are static, I think we should drop "expr_" prefix for > >>> them. Not in this patch, though. > >> > >> If functions take Expr as an argument like these do, they could be > >> considered methods of Expr. In that case dropping the expr_ prefix would > >> violate our naming convention. It is not about static or global here. > >> > >> As an alternative they could be considered as methods of Parse, but > >> then they would need to have parse_ prefix. > >> > >> For 'code' vs 'emit' - 'code' is fine by me as long as it is static. But > >> if it goes public, then either 'code' or 'emit' must be chosen as one > >> correct suffix. Not a mix. > >> > > After some thought, I think you are right. However, I would suggest removing the > > parser and vdbe from these functions and converting them to proper struct expr > > methods. This way we can make these functions return a value (most likely as an > > "out" argument). For example expr_code_dec() should give us DECIMAL. In this > > case we can make some improvements, for example we can remove "is_neg" from > > expr_code_int() and turn it into expr_code_uint(), since we know that this '-' > > sign will be specified as another expr. Also, since these will be valid expr > > methods, we can drop "static" from their definition. We then should name them > > accordingly, for example "expr_code_dec" may be named "expr_to_dec". > > AFAIU, their value is not only in converting the value, but also in doing > the sqlVdbeAddOp action. You will need to duplicate this call in all places > where expr_to_dec() would be used. The aspiration for refactoring this code > is righteous anyway though. > > > diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c > > index 55e494332..86de3f98a 100644 > > --- a/src/box/sql/vdbe.c > > +++ b/src/box/sql/vdbe.c > > @@ -1438,6 +1438,26 @@ case OP_Array: { > > break; > > } > > > > +/** > > + * Opcode: Map P1 P2 P3 * * > > + * Synopsis: r[P2] = map(P3@P1) > > + * > > + * Construct an MAP value from P1 registers starting at reg(P3). > > + */ > > +case OP_Map: { > > + pOut = &aMem[pOp->p2]; > > + > > + uint32_t size; > > + struct region *region = &fiber()->gc; > > + size_t svp = region_used(region); > > + char *val = mem_encode_map(&aMem[pOp->p3], pOp->p1, &size, region); > > + if (val == NULL || mem_copy_map(pOut, val, size) != 0) { > > + region_truncate(region, svp); > > + goto abort_due_to_error; > > + } > > You should probably truncate the region regardless of the result. Otherwise > in case of success you will leak the region inside of the query bit by bit > while SELECT works: > > box.execute('CREATE TABLE test (id INTEGER PRIMARY KEY)') > box.execute('INSERT INTO test VALUES (1), (2), (3)') > box.execute('SELECT {id: id} FROM test') > > Here you will do OP_Map 3 times, all will leave some region memory leaked > every time. It is freed in the end of execution probably, but it might do > some big usage while the request is in progress when the row count is much > bigger than 3. > Thank you! Fixed. > Btw, worth adding a multirow test. All current map tests select a single row. There are several tests where the number of selected rows is more than one, for example map-3 or map-4, but the number of rows is still quite low. I added another test that selects 1000 rows with maps. Diff: diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 6824f11bf..33207db6b 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -1456,6 +1456,7 @@ case OP_Map: { region_truncate(region, svp); goto abort_due_to_error; } + region_truncate(region, svp); break; } diff --git a/test/sql-tap/map.test.lua b/test/sql-tap/map.test.lua index 7791ca779..c51880cfd 100755 --- a/test/sql-tap/map.test.lua +++ b/test/sql-tap/map.test.lua @@ -1,6 +1,6 @@ #!/usr/bin/env tarantool local test = require("sqltester") -test:plan(131) +test:plan(132) box.schema.func.create('M1', { language = 'Lua', @@ -1165,6 +1165,18 @@ test:do_test( "[{5: 1}]" }) +-- Make sure that multiple row with maps can be selected properly. +local maps = {{a1 = 1}} +local strs = "({'a1': 1})" +for i = 2, 1000 do maps[i] = {['a'..tostring(i)] = i} end +for i = 2, 1000 do strs = strs .. string.format(", ({'a%d': %d})", i, i) end +test:do_execsql_test( + "map-15", + [[ + SELECT * FROM (VALUES]]..strs..[[); + ]], maps) + + box.execute([[DROP TABLE t1;]]) box.execute([[DROP TABLE t;]])