From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp60.i.mail.ru (smtp60.i.mail.ru [217.69.128.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 670F4469719 for ; Tue, 29 Sep 2020 02:29:27 +0300 (MSK) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) From: Roman Khabibov In-Reply-To: <20200916133024.GC10599@tarantool.org> Date: Tue, 29 Sep 2020 02:29:26 +0300 Content-Transfer-Encoding: quoted-printable Message-Id: <8F4523FC-8185-468D-A784-C3A81EFF32C8@tarantool.org> References: <20200911215115.6622-1-roman.habibov@tarantool.org> <20200911215115.6622-5-roman.habibov@tarantool.org> <20200916133024.GC10599@tarantool.org> Subject: Re: [Tarantool-patches] [PATCH v4 4/6] sql: use parser's region of "index" array List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Nikita Pettik Cc: tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org Hi! Thanks for the review. > On Sep 16, 2020, at 16:30, Nikita Pettik = wrote: >=20 > On 12 Sep 00:51, Roman Khabibov wrote: >> Allocate memory for the "index" array of ephemeral space on the >> parser's region instead of a heap as it was before. Fixed a memory >> leak that realloc() generated. >>=20 >> Needed for #3075 >=20 > Why it is related to 3075?? Fixed to -> 2349. Due to this patch, I avoid malloc inside = sql_space_shallow_copy. >> --- >> src/box/sql/build.c | 42 +++++++++++++++++++++++++++++++----------- >> 1 file changed, 31 insertions(+), 11 deletions(-) >>=20 >> diff --git a/src/box/sql/build.c b/src/box/sql/build.c >> index 6cc76bb77..d1d240315 100644 >> --- a/src/box/sql/build.c >> +++ b/src/box/sql/build.c >> @@ -2226,29 +2226,49 @@ vdbe_emit_new_sec_index_id(struct Parse = *parse, uint32_t space_id, >> } >>=20 >> /** >> - * Add new index to table's indexes list. >> + * Add new index to ephemeral @a space's "index" array. Reallocate >> + * memory on @a region if needed. >> + * >> * We follow convention that PK comes first in list. >> * >> + * @param region Region (on parser). >> * @param space Space to which belongs given index. >> * @param index Index to be added to list. >> */ >> static void >> -table_add_index(struct space *space, struct index *index) >> +sql_space_add_index(struct region *region, struct space *space, >> + struct index *index) >> { >> uint32_t idx_count =3D space->index_count; >> - size_t indexes_sz =3D sizeof(struct index *) * (idx_count + 1); >> - struct index **idx =3D (struct index **) realloc(space->index, >> - indexes_sz); >> - if (idx =3D=3D NULL) { >> - diag_set(OutOfMemory, indexes_sz, "realloc", "idx"); >> - return; >> - } >> - space->index =3D idx; >> + size_t size =3D 0; >> + struct index **idx =3D NULL; >> + if (idx_count =3D=3D 0) { >> + idx =3D region_alloc_array(region, typeof(struct index = *), 1, >> + &size); >> + if (idx =3D=3D NULL) >> + goto alloc_error; >> + /* >> + * Reallocate each time the idx_count becomes equal to the >> + * power of two. >> + */ >> + } else if ((idx_count & (idx_count - 1)) =3D=3D 0) { >> + idx =3D region_alloc_array(region, typeof(struct index = *), >> + idx_count * 2, &size); >> + if (idx =3D=3D NULL) >> + goto alloc_error; >> + memcpy(idx, space->index, idx_count * sizeof(struct = index *)); >> + } >> + if (idx !=3D NULL) >> + space->index =3D idx; >=20 > DOes it make any sense? Why not simply call ::free (in case there = really > was memleak)? It was a memleak, really. I checked it by valgrind. This is the main reason. So that when destroying the parser, you don't = have to check whether it was CREATE or ALTER, whether malloc or realloc was = called inside build.c, and don't call free. The parser region is well suited = for this task. >> /* Make sure that PK always comes as first member. */ >> if (index->def->iid =3D=3D 0 && idx_count !=3D 0) >> SWAP(space->index[0], index); >> space->index[space->index_count++] =3D index; >> space->index_id_max =3D MAX(space->index_id_max, = index->def->iid);; >> + return; >> + >> +alloc_error: >> + diag_set(OutOfMemory, size, "region_alloc_array", "idx"); >> } >>=20 >> int >> @@ -2717,7 +2737,7 @@ sql_create_index(struct Parse *parse) { >>=20 >> if (tbl_name !=3D NULL) >> goto exit_create_index; >> - table_add_index(space, index); >> + sql_space_add_index(&parse->region, space, index); >> index =3D NULL; >>=20 >> /* Clean up before exiting. */ >> --=20 >> 2.24.3 (Apple Git-128) >>=20 commit 177c4389af647677afcce9fa58b12f00e398fbe9 Author: Roman Khabibov Date: Thu Sep 3 14:26:47 2020 +0300 sql: use parser's region of "index" array =20 Allocate memory for the "index" array of ephemeral space on the parser's region instead of a heap as it was before. Fixed a memory leak that realloc() generated. =20 Needed for #2349 diff --git a/src/box/sql/build.c b/src/box/sql/build.c index 6cc76bb77..d1d240315 100644 --- a/src/box/sql/build.c +++ b/src/box/sql/build.c @@ -2226,29 +2226,49 @@ vdbe_emit_new_sec_index_id(struct Parse *parse, = uint32_t space_id, } =20 /** - * Add new index to table's indexes list. + * Add new index to ephemeral @a space's "index" array. Reallocate + * memory on @a region if needed. + * * We follow convention that PK comes first in list. * + * @param region Region (on parser). * @param space Space to which belongs given index. * @param index Index to be added to list. */ static void -table_add_index(struct space *space, struct index *index) +sql_space_add_index(struct region *region, struct space *space, + struct index *index) { uint32_t idx_count =3D space->index_count; - size_t indexes_sz =3D sizeof(struct index *) * (idx_count + 1); - struct index **idx =3D (struct index **) realloc(space->index, - indexes_sz); - if (idx =3D=3D NULL) { - diag_set(OutOfMemory, indexes_sz, "realloc", "idx"); - return; - } - space->index =3D idx; + size_t size =3D 0; + struct index **idx =3D NULL; + if (idx_count =3D=3D 0) { + idx =3D region_alloc_array(region, typeof(struct index = *), 1, + &size); + if (idx =3D=3D NULL) + goto alloc_error; + /* + * Reallocate each time the idx_count becomes equal to the + * power of two. + */ + } else if ((idx_count & (idx_count - 1)) =3D=3D 0) { + idx =3D region_alloc_array(region, typeof(struct index = *), + idx_count * 2, &size); + if (idx =3D=3D NULL) + goto alloc_error; + memcpy(idx, space->index, idx_count * sizeof(struct = index *)); + } + if (idx !=3D NULL) + space->index =3D idx; /* Make sure that PK always comes as first member. */ if (index->def->iid =3D=3D 0 && idx_count !=3D 0) SWAP(space->index[0], index); space->index[space->index_count++] =3D index; space->index_id_max =3D MAX(space->index_id_max, = index->def->iid);; + return; + +alloc_error: + diag_set(OutOfMemory, size, "region_alloc_array", "idx"); } =20 int @@ -2717,7 +2737,7 @@ sql_create_index(struct Parse *parse) { =20 if (tbl_name !=3D NULL) goto exit_create_index; - table_add_index(space, index); + sql_space_add_index(&parse->region, space, index); index =3D NULL; =20 /* Clean up before exiting. */