From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id F03786FC8F; Thu, 25 Mar 2021 00:29:09 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org F03786FC8F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1616621350; bh=X0ZQtSGtRISdrUuU6OJWe0hmSLK2nqSrey44vsRygwQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=RxkT6jHa/ej7MDrgQGJBpw2Qt/kbXc7V8iNmzcjWlH1eOWtT0Y7raV2Eo3ypj43sb BWQlMfz8E18KD42sUVHApQOaS7gT4n1fgBvLnAssoeB89kpPYIvPNzKi9Wl5+FJ1P4 JZyALTYGnZH4ZoeTn2x/vJM9OacxAnR8Z95azsWU= Received: from smtp48.i.mail.ru (smtp48.i.mail.ru [94.100.177.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 4E22F6B469 for ; Thu, 25 Mar 2021 00:24:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 4E22F6B469 Received: by smtp48.i.mail.ru with esmtpa (envelope-from ) id 1lPAze-0004ib-KY; Thu, 25 Mar 2021 00:24:47 +0300 To: tarantool-patches@dev.tarantool.org, kyukhin@tarantool.org Date: Wed, 24 Mar 2021 22:24:29 +0100 Message-Id: <725ea2e651c6e29a33d43bcee73f4e3f144d76c7.1616620860.git.v.shpilevoy@tarantool.org> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9064ADF4728AA0EE9AECA9F3C9C9885BEE78E91CF33279E24182A05F5380850408B22CED2333E53BAD3C19D1B8201F8B88EA60DCEC279961703381A919034B6AB X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE71107D7B19CDFFE90EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006377CC130305260E47D8638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95C5DD32608FC869F5DF148B6E867E3E9E19044C7739D4ED6EBA471835C12D1D9774AD6D5ED66289B5278DA827A17800CE77A825AB47F0FC8649FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C34C82C86BFC697D19117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CFF035437B6AEB70F476E601842F6C81A12EF20D2F80756B5F7E9C4E3C761E06A776E601842F6C81A127C277FBC8AE2E8BCE107276B2AA573D3AA81AA40904B5D9DBF02ECDB25306B2201CA6A4E26CD07C3BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B091A620F70A64A45A99449624AB7ADAF372E808ACE2090B5E1725E5C173C3A84C33D321E7403792E343D7D993A0B92D134F5D81C698A659EA775ECD9A6C639B01BC09775C1D3CA48CFBE90F13D913F449135872C767BF85DA22EF20D2F80756B5F40A5AABA2AD3711975ECD9A6C639B01B78DA827A17800CE76D0F27F7E6A6C418731C566533BA786A40A5AABA2AD371193C9F3DD0FB1AF5EB417FD7EC7EC0BD913C9F3DD0FB1AF5EB4E70A05D1297E1BBCB5012B2E24CD356 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2368A440D3B0F6089093C9A16E5BC824AC8B6CDF511875BC4E8F7B195E1C97831148145A0506FFD481BBFD1EA3789D373 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C4B5F30041107ECCAEB459AD25B48836B76E15E3BE9FD72F9AC6CDE5D1141D2B1C82594762DF9190BC6D6D7B71B550D1D53BC048D4545F93EEAD91A466A1DEF99B296C473AB1E14218DA371E42557AFC06296C473AB1E142186B023E84F73EF47C6DABF04D5057A81F728CF7B057D10C709E541A154B51D14BB936CB490224F2464EEA7BD89490CAC0EDDA962BC3F61961 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A1AF43396E40A36F84E2BC36E985AEF5D212EE68175A78B7B679B689D5F1C3A7E9334FCCB38891C31D7E09C32AA3244C54FDE34B2EE8316032209DC1ACAE107F9CA7333006C390A0729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojjqzNotmU+gdtq9H1TCjMjA== X-Mailru-Sender: 504CC1E875BF3E7D9BC0E5172ADA3110A906815429F9D9604275BC8943AB7C44197747113D9E358107784C02288277CA03E0582D3806FB6A5317862B1921BA260ED6CFD6382C13A6112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH 03/15] tuple: pass global ibuf explicitly where possible X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Vladislav Shpilevoy via Tarantool-patches Reply-To: Vladislav Shpilevoy Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Code in lua/tuple.c used global tarantool_lua_ibuf in many places relying on it never being changed and not reused by other code until a yield. But it is not so. In fact, as it was discovered in #5632, in any Lua function may be started GC. Any GC handler might touch some API also using tarantool_lua_ibuf inside. This makes the first usage in lua/tuple.c invalid - the buffer could be reset or reallocated or its wpos/rpos could change during GC. In order to fix this, first of all there should be clear points where the buffer is taken, and where it becomes not needed anymore. The patch makes code in lua/tuple.c take tarantool_lua_ibuf when it is needed first time. Not during usage. The same is done for the fiber region for the API symmetry. Part of #5632 (cherry picked from commit fabf0d57ea31d69be848fa33c98397a9a4a964e9) --- src/box/lua/tuple.c | 60 ++++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 23 deletions(-) diff --git a/src/box/lua/tuple.c b/src/box/lua/tuple.c index 617acc9aa..00ac3b44c 100644 --- a/src/box/lua/tuple.c +++ b/src/box/lua/tuple.c @@ -134,10 +134,8 @@ luaT_istuple(struct lua_State *L, int narg) * Helper for (). */ static int -luaT_tuple_encode_values(struct lua_State *L) +luaT_tuple_encode_values(struct lua_State *L, struct ibuf *buf) { - struct ibuf *buf = tarantool_lua_ibuf; - ibuf_reset(buf); struct mpstream stream; mpstream_init(&stream, buf, ibuf_reserve_cb, ibuf_alloc_cb, luamp_error, L); @@ -150,22 +148,31 @@ luaT_tuple_encode_values(struct lua_State *L) return 0; } -typedef void luaT_mpstream_init_f(struct mpstream *stream, struct lua_State *L); +typedef void luaT_mpstream_init_f(struct mpstream *stream, struct lua_State *L, + void *buffer); static void -luaT_mpstream_init_lua_ibuf(struct mpstream *stream, struct lua_State *L) +luaT_mpstream_init_lua_ibuf(struct mpstream *stream, struct lua_State *L, + void *buffer) { - mpstream_init(stream, tarantool_lua_ibuf, ibuf_reserve_cb, + mpstream_init(stream, (struct ibuf *)buffer, ibuf_reserve_cb, ibuf_alloc_cb, luamp_error, L); } static void -luaT_mpstream_init_box_region(struct mpstream *stream, struct lua_State *L) +luaT_mpstream_init_box_region(struct mpstream *stream, struct lua_State *L, + void *buffer) { - mpstream_init(stream, &fiber()->gc, region_reserve_cb, region_alloc_cb, - luamp_error, L); + mpstream_init(stream, (struct region *)buffer, region_reserve_cb, + region_alloc_cb, luamp_error, L); } +/** Helper to pass parameters into luaT_tuple_encode_table via the Lua stack. */ +struct luaT_tuple_encode_ctx { + luaT_mpstream_init_f *mpstream_init_f; + void *buffer; +}; + /** * Encode a Lua table or a tuple as MsgPack. * @@ -177,8 +184,8 @@ static int luaT_tuple_encode_table(struct lua_State *L) { struct mpstream stream; - luaT_mpstream_init_f *luaT_mpstream_init_f = lua_topointer(L, 1); - luaT_mpstream_init_f(&stream, L); + const struct luaT_tuple_encode_ctx *ctx = lua_topointer(L, 1); + ctx->mpstream_init_f(&stream, L, ctx->buffer); luamp_encode_tuple(L, &tuple_serializer, &stream, 2); mpstream_flush(&stream); return 0; @@ -189,7 +196,8 @@ luaT_tuple_encode_table(struct lua_State *L) */ static int luaT_tuple_encode_on_mpstream(struct lua_State *L, int idx, - luaT_mpstream_init_f *luaT_mpstream_init_f) + luaT_mpstream_init_f *luaT_mpstream_init_f, + void *buffer) { assert(idx != 0); if (!lua_istable(L, idx) && !luaT_istuple(L, idx)) { @@ -212,7 +220,11 @@ luaT_tuple_encode_on_mpstream(struct lua_State *L, int idx, lua_rawgeti(L, LUA_REGISTRYINDEX, luaT_tuple_encode_table_ref); assert(lua_isfunction(L, -1)); - lua_pushlightuserdata(L, luaT_mpstream_init_f); + struct luaT_tuple_encode_ctx ctx = { + .mpstream_init_f = luaT_mpstream_init_f, + .buffer = buffer, + }; + lua_pushlightuserdata(L, &ctx); lua_pushvalue(L, idx); int rc = luaT_call(L, 2, 0); @@ -225,12 +237,11 @@ luaT_tuple_encode_on_mpstream(struct lua_State *L, int idx, */ static char * luaT_tuple_encode_on_lua_ibuf(struct lua_State *L, int idx, - size_t *tuple_len_ptr) + size_t *tuple_len_ptr, struct ibuf *buf) { - struct ibuf *buf = tarantool_lua_ibuf; - ibuf_reset(buf); if (luaT_tuple_encode_on_mpstream(L, idx, - luaT_mpstream_init_lua_ibuf) != 0) + luaT_mpstream_init_lua_ibuf, + buf) != 0) return NULL; if (tuple_len_ptr != NULL) *tuple_len_ptr = ibuf_used(buf); @@ -246,7 +257,8 @@ luaT_tuple_encode(struct lua_State *L, int idx, size_t *tuple_len_ptr) struct region *region = &fiber()->gc; size_t region_svp = region_used(region); if (luaT_tuple_encode_on_mpstream(L, idx, - luaT_mpstream_init_box_region) != 0) { + luaT_mpstream_init_box_region, + region) != 0) { region_truncate(region, region_svp); return NULL; } @@ -267,15 +279,16 @@ luaT_tuple_encode(struct lua_State *L, int idx, size_t *tuple_len_ptr) box_tuple_t * luaT_tuple_new(struct lua_State *L, int idx, box_tuple_format_t *format) { + struct ibuf *ibuf = tarantool_lua_ibuf; + ibuf_reset(ibuf); size_t tuple_len; - char *tuple_data = luaT_tuple_encode_on_lua_ibuf(L, idx, &tuple_len); + char *tuple_data = luaT_tuple_encode_on_lua_ibuf(L, idx, &tuple_len, + ibuf); if (tuple_data == NULL) return NULL; box_tuple_t *tuple = box_tuple_new(format, tuple_data, tuple_data + tuple_len); - if (tuple == NULL) - return NULL; - ibuf_reinit(tarantool_lua_ibuf); + ibuf_reinit(ibuf); return tuple; } @@ -295,7 +308,8 @@ lbox_tuple_new(lua_State *L) box_tuple_format_t *fmt = box_tuple_format_default(); if (argc != 1 || (!lua_istable(L, 1) && !luaT_istuple(L, 1))) { struct ibuf *buf = tarantool_lua_ibuf; - luaT_tuple_encode_values(L); /* may raise */ + ibuf_reset(buf); + luaT_tuple_encode_values(L, buf); /* may raise */ struct tuple *tuple = box_tuple_new(fmt, buf->buf, buf->buf + ibuf_used(buf)); ibuf_reinit(buf); -- 2.24.3 (Apple Git-128)