From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 94B806EC5F; Thu, 15 Apr 2021 23:40:00 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 94B806EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1618519200; bh=B9IGVxWimTDblzjavgm3Mkuct7IzC+iuJJueXKToRJo=; h=In-Reply-To:Date:Cc:References:To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=tDsOt1wAXjeceEddd14STQe8o3Bn5kbFsnhNBbvnewywZASJRxvsTar1xs7Mg2J+3 OplY314ejaNLDGs+D6hH7sMNtX3FUVMaM05Lm1eJjVbCCzttFz5HwF6GKZQ49lMnRC 9EM46VBQqJP0BGlta35e+xrEtNy4KT4/FgrVVy0U= Received: from smtp35.i.mail.ru (smtp35.i.mail.ru [94.100.177.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id ED1FF6EC5F for ; Thu, 15 Apr 2021 23:39:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org ED1FF6EC5F Received: by smtp35.i.mail.ru with esmtpa (envelope-from ) id 1lX8mL-000763-Oj; Thu, 15 Apr 2021 23:39:58 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) In-Reply-To: <9edf496a-3681-44f9-6e89-1d44bcc193b2@tarantool.org> Date: Fri, 16 Apr 2021 01:39:55 +0500 Cc: tml Content-Transfer-Encoding: quoted-printable Message-Id: References: <20210311054949.91263-1-roman.habibov@tarantool.org> <9edf496a-3681-44f9-6e89-1d44bcc193b2@tarantool.org> To: Sergey Bronnikov X-Mailer: Apple Mail (2.3608.120.23.2.1) X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92FFCB8E6708E7480257C85EA0BB7A95D5E28B957962BB550182A05F538085040A2B05C3231EAC3971787CBF78610895B39AF7683BFC208E11A314520AFCCD7C4 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7BB17EE3498E810FEEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637B6B039AA0E19584A8638F802B75D45FF914D58D5BE9E6BC1A93B80C6DEB9DEE97C6FB206A91F05B2C30F496B74459189578E015236847657C7C2BEF539CC105AD2E47CDBA5A96583C09775C1D3CA48CFCA5A41EBD8A3A0199FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C317B107DEF921CE79117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CF5A8B64BA16C9E34CBA3038C0950A5D36C8A9BA7A39EFB766EC990983EF5C0329BA3038C0950A5D36D5E8D9A59859A8B6D7952DD32F4FD77C76E601842F6C81A1F004C906525384307823802FF610243D2EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE71D0063F52110EA4A731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A2368A440D3B0F6089093C9A16E5BC824A2A04A2ABAA09D25379311020FFC8D4ADD10B2DE89DD7797157CC7B98B2D16E1D X-C1DE0DAB: 0D63561A33F958A5CCAEA69F8C1822F16BA019C7D1097A3F4214FABE39D97148D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3468964757141B82E02EF7ABD78EFE337EE9DB6E12EFBDCE7EAE0C51C3E7BB6275F2F4A3DAEBC8BC481D7E09C32AA3244C4C082DB2E414DBAA46E340D60AF828651E098CBE561D6343729B2BEF169E0186 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj3S6P1v0GIqSIxp0RLCJ3jQ== X-Mailru-Sender: 3E0F75B73734E6D91F8A0178718F87770D41D44FD6ABF88EABBB7376667DB10D48429B3ABF0D9D9E5B7A0EB1374FC74C72D6B4FCE48DF6482536B595554F027CD993E5A2F485DEE59437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH] serializer: serialize recursive structures X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Roman Khabibov via Tarantool-patches Reply-To: Roman Khabibov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Hi! Thanks for the review. > On Apr 13, 2021, at 20:54, Sergey Bronnikov = wrote: >=20 > Hello! >=20 > thanks for the patch! >=20 > 1. I have reverted all changes and test passed. I dropped this tests. > 2. In commit message you declares recursive structures support. > What is the max supported depth of recursive structure? > I believe it is worth to describe possible limitations in commit = message. Hmm, I think this term =E2=80=98depth' (the number of nested tables) is = not related to this patch. For example, setmetatable({}, {__serialize =3D function(a) return = {{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}= }}}}}}}}}}}} end}) This case works on master. The number of nested tables is defined by = size of allocated memory. > See inline >=20 > On 11.03.2021 08:49, Roman Khabibov via Tarantool-patches wrote: >> Fix bug with bus error during serializing of recursive >> structures. >>=20 >> Closes #3228 >> --- >>=20 >> Branch: = https://github.com/tarantool/tarantool/tree/romanhabibov/gh-3228-visited-s= et >> Issue: https://github.com/tarantool/tarantool/issues/3228 >>=20 >> src/box/lua/call.c | 6 +- >> src/box/lua/execute.c | 2 +- >> src/box/lua/serialize_lua.c | 96 ++------ >> src/box/lua/tuple.c | 2 +- >> src/box/sql/func.c | 2 +- >> src/lua/msgpack.c | 10 +- >> src/lua/pickle.c | 2 +- >> src/lua/utils.c | 226 = ++++++++++++++++-- >> src/lua/utils.h | 43 +++- >> ...-3228-serializer-look-for-recursion.result | 67 ++++++ >> ...228-serializer-look-for-recursion.test.lua | 17 ++ >> test/swim/swim.result | 18 +- >> third_party/lua-cjson/lua_cjson.c | 4 +- >> third_party/lua-yaml/lyaml.cc | 88 +++---- >> 14 files changed, 390 insertions(+), 193 deletions(-) >> create mode 100644 = test/app/gh-3228-serializer-look-for-recursion.result >> create mode 100644 = test/app/gh-3228-serializer-look-for-recursion.test.lua >=20 > >=20 >> diff --git a/test/app/gh-3228-serializer-look-for-recursion.test.lua = b/test/app/gh-3228-serializer-look-for-recursion.test.lua >> new file mode 100644 >> index 000000000..1c9b0375f >> --- /dev/null >> +++ b/test/app/gh-3228-serializer-look-for-recursion.test.lua >> @@ -0,0 +1,17 @@ >> +test_run =3D require('test_run').new() >> + >> +-- >> +-- gh-3228: Check the error message in the case of a __serialize >> +-- function generating infinite recursion. >> +-- >> +setmetatable({}, {__serialize =3D function(a) return a end}) >> +setmetatable({}, {__serialize =3D function(a) return {a} end}) >> +setmetatable({}, {__serialize =3D function(a) return {a, a} end}) >> +setmetatable({}, {__serialize =3D function(a) return {a, a, a} end}) >> +setmetatable({}, {__serialize =3D function(a) return {{a, a}, a} = end}) >> +setmetatable({}, {__serialize =3D function(a) return {a, 1} end}) >> +setmetatable({}, {__serialize =3D function(a) return {{{{a}}}} end}) >> +setmetatable({}, {__serialize =3D function(a) return {{{{1}}}} end}) >=20 > 3. Are you sure 4 levels depth is enough to check recursive structure? >=20 > What is about testing max depth? ^^^ >=20 >> +b =3D {} >> +setmetatable({b}, {__serialize =3D function(a) return {a_1 =3D a, = a_2 =3D a, b_1 =3D b, b_2 =3D b} end}) >> +setmetatable({b}, {__serialize =3D function(a) return {a_1 =3D a, = a_2 =3D {a, b}, b =3D b} end}) >> diff --git a/test/swim/swim.result b/test/swim/swim.result >> index bfc83c143..539131677 100644 >> --- a/test/swim/swim.result >> +++ b/test/swim/swim.result >> @@ -1322,16 +1322,13 @@ m_list >> incarnation: cdata {generation =3D 0ULL, version =3D 1ULL} >> uuid: 00000000-0000-1000-8000-000000000002 >> payload_size: 0 >> - - uri: 127.0.0.1: >> - status: alive >> - incarnation: cdata {generation =3D 0ULL, version =3D 2ULL} >> - uuid: 00000000-0000-1000-8000-000000000001 >> - payload_size: 8 >> - - uri: 127.0.0.1: >> + - &0 >> + uri: 127.0.0.1: >> status: alive >> incarnation: cdata {generation =3D 0ULL, version =3D 2ULL} >> uuid: 00000000-0000-1000-8000-000000000001 >> payload_size: 8 >> + - *0 >> ... >> e_list >> --- >> @@ -1374,16 +1371,13 @@ fiber.sleep(0) >> -- Two events - status update to 'left', and 'drop'. >> m_list >> --- >> -- - uri: 127.0.0.1: >> - status: left >> - incarnation: cdata {generation =3D 0ULL, version =3D 1ULL} >> - uuid: 00000000-0000-1000-8000-000000000002 >> - payload_size: 0 >> - - uri: 127.0.0.1: >> +- - &0 >> + uri: 127.0.0.1: >> status: left >> incarnation: cdata {generation =3D 0ULL, version =3D 1ULL} >> uuid: 00000000-0000-1000-8000-000000000002 >> payload_size: 0 >> + - *0 >> ... >> e_list >> --- >=20 > 4. How changes in swim test related to patch? >=20 > It is not clear from commit message. See new commit message. > commit 7206b41ff1456e71474c2ff84b214f770e5897af Author: Roman Khabibov Date: Fri Feb 5 16:55:56 2021 +0300 serializer: serialize recursive structures =20 Fix bug with bus error during serializing of recursive structures and always print aliases if node become referenced before or after serialization. =20 This patch adds map with serialized nodes and results. The map is used to mark referenced nodes which appears during serializing. =20 Closes #3228 =20 @TarantoolBot __serialize parameter If __serialize parameter is initialized as a function then this function should be pure. A pure function is a function which returns identical values for identical arguments (no variation with local static variables, non-local variables, mutable reference arguments or input streams). Otherwise, the behavior is undefined. diff --git a/changelogs/unreleased/serializer-bug.md = b/changelogs/unreleased/serializer-bug.md new file mode 100755 index 000000000..96f359d64 --- /dev/null +++ b/changelogs/unreleased/serializer-bug.md @@ -0,0 +1,4 @@ +## bugfix/core + +* Recursive structures appeared after __serialize call don't lead +* to segfault anymore in lua and yaml serializers (gh-5392). diff --git a/src/box/lua/call.c b/src/box/lua/call.c index 0315e720c..e4907a3f7 100644 --- a/src/box/lua/call.c +++ b/src/box/lua/call.c @@ -193,7 +193,7 @@ luamp_encode_call_16(lua_State *L, struct = luaL_serializer *cfg, */ for (int i =3D 1; i <=3D nrets; ++i) { struct luaL_field field; - if (luaL_tofield(L, cfg, NULL, i, &field) < 0) + if (luaL_tofield(L, cfg, 0, NULL, i, &field) < = 0) return luaT_error(L); struct tuple *tuple; if (field.type =3D=3D MP_EXT && @@ -222,7 +222,7 @@ luamp_encode_call_16(lua_State *L, struct = luaL_serializer *cfg, * Inspect the first result */ struct luaL_field root; - if (luaL_tofield(L, cfg, NULL, 1, &root) < 0) + if (luaL_tofield(L, cfg, 0, NULL, 1, &root) < 0) return luaT_error(L); struct tuple *tuple; if (root.type =3D=3D MP_EXT && (tuple =3D luaT_istuple(L, 1)) !=3D= NULL) { @@ -252,7 +252,7 @@ luamp_encode_call_16(lua_State *L, struct = luaL_serializer *cfg, for (uint32_t t =3D 1; t <=3D root.size; t++) { lua_rawgeti(L, 1, t); struct luaL_field field; - if (luaL_tofield(L, cfg, NULL, -1, &field) < 0) + if (luaL_tofield(L, cfg, 0, NULL, -1, &field) < 0) return luaT_error(L); if (field.type =3D=3D MP_EXT && (tuple =3D = luaT_istuple(L, -1))) { tuple_to_mpstream(tuple, stream); diff --git a/src/box/lua/execute.c b/src/box/lua/execute.c index 926a0a61c..76d271f5c 100644 --- a/src/box/lua/execute.c +++ b/src/box/lua/execute.c @@ -328,7 +328,7 @@ lua_sql_bind_decode(struct lua_State *L, struct = sql_bind *bind, int idx, int i) bind->name =3D NULL; bind->name_len =3D 0; } - if (luaL_tofield(L, luaL_msgpack_default, NULL, -1, &field) < 0) + if (luaL_tofield(L, luaL_msgpack_default, 0, NULL, -1, &field) < = 0) return -1; switch (field.type) { case MP_UINT: diff --git a/src/box/lua/serialize_lua.c b/src/box/lua/serialize_lua.c index caa08a60f..17b434ba5 100644 --- a/src/box/lua/serialize_lua.c +++ b/src/box/lua/serialize_lua.c @@ -115,6 +115,7 @@ struct lua_dumper { /** Anchors for self references */ int anchortable_index; unsigned int anchor_number; + int serialized_objs_idx; =20 /** Error message buffer */ char err_msg[256]; @@ -215,7 +216,7 @@ trace_node(struct lua_dumper *d) struct luaL_field field; =20 memset(&field, 0, sizeof(field)); - luaL_checkfield(d->L, d->cfg, lua_gettop(d->L), &field); + luaL_checkfield(d->L, d->cfg, 0, lua_gettop(d->L), &field); =20 if (field.type < lengthof(mp_type_names)) { if (field.type =3D=3D MP_EXT) { @@ -234,7 +235,7 @@ trace_node(struct lua_dumper *d) =20 memset(&field, 0, sizeof(field)); =20 - luaL_checkfield(d->L, d->cfg, top, &field); + luaL_checkfield(d->L, d->cfg, 0, top, &field); say_info("serializer-trace: node :\tfield type %s (%d)", type_str, field.type); } @@ -339,41 +340,16 @@ emit_node(struct lua_dumper *d, struct node *nd, = int indent, static const char * get_lua_anchor(struct lua_dumper *d) { - const char *s =3D ""; - - lua_pushvalue(d->L, -1); - lua_rawget(d->L, d->anchortable_index); - if (!lua_toboolean(d->L, -1)) { - lua_pop(d->L, 1); - return NULL; + const char *anchor =3D NULL; + int code =3D get_anchor(d->L, d->anchortable_index, = &d->anchor_number, + &anchor); + if (code =3D=3D GET_ANCHOR_NAMED_NOT) { + trace_anchor(anchor, false); + } else if (code =3D=3D GET_ANCHOR_ALIASED) { + trace_anchor(anchor, true); + return ""; } - - if (lua_isboolean(d->L, -1)) { - /* - * This element is referenced more - * than once but has not been named. - */ - char buf[32]; - snprintf(buf, sizeof(buf), "%u", d->anchor_number++); - lua_pop(d->L, 1); - lua_pushvalue(d->L, -1); - lua_pushstring(d->L, buf); - s =3D lua_tostring(d->L, -1); - lua_rawset(d->L, d->anchortable_index); - trace_anchor(s, false); - } else { - /* - * An aliased element. - * - * FIXME: Need an example to use. - * - * const char *str =3D lua_tostring(d->L, -1); - */ - const char *str =3D lua_tostring(d->L, -1); - trace_anchor(str, true); - lua_pop(d->L, 1); - } - return s; + return anchor; } =20 static void @@ -783,7 +759,7 @@ dump_node(struct lua_dumper *d, struct node *nd, int = indent) return -1; =20 memset(field, 0, sizeof(*field)); - luaL_checkfield(d->L, d->cfg, lua_gettop(d->L), field); + luaL_checkfield(d->L, d->cfg, 0, lua_gettop(d->L), field); =20 switch (field->type) { case MP_NIL: @@ -881,49 +857,6 @@ dump_node(struct lua_dumper *d, struct node *nd, = int indent) return emit_node(d, nd, indent, str, len); } =20 -/** - * Find references to tables, we use it - * to find self references in tables. - */ -static void -find_references(struct lua_dumper *d) -{ - int newval; - - if (lua_type(d->L, -1) !=3D LUA_TTABLE) - return; - - /* Copy of a table for self refs */ - lua_pushvalue(d->L, -1); - lua_rawget(d->L, d->anchortable_index); - if (lua_isnil(d->L, -1)) - newval =3D 0; - else if (!lua_toboolean(d->L, -1)) - newval =3D 1; - else - newval =3D -1; - lua_pop(d->L, 1); - - if (newval !=3D -1) { - lua_pushvalue(d->L, -1); - lua_pushboolean(d->L, newval); - lua_rawset(d->L, d->anchortable_index); - } - - if (newval !=3D 0) - return; - - /* - * Other values and keys in the table - */ - lua_pushnil(d->L); - while (lua_next(d->L, -2) !=3D 0) { - find_references(d); - lua_pop(d->L, 1); - find_references(d); - } -} - /** * Dump recursively from the root node. */ @@ -935,7 +868,8 @@ dump_root(struct lua_dumper *d) }; int ret; =20 - luaL_checkfield(d->L, d->cfg, lua_gettop(d->L), &nd.field); + luaL_checkfield(d->L, d->cfg, d->serialized_objs_idx, + lua_gettop(d->L), &nd.field); =20 if (nd.field.type !=3D MP_ARRAY || nd.field.size !=3D 1) { d->err =3D EINVAL; @@ -982,9 +916,13 @@ lua_encode(lua_State *L, struct luaL_serializer = *serializer, dumper.anchortable_index =3D lua_gettop(L); dumper.anchor_number =3D 0; =20 + lua_newtable(L); + dumper.serialized_objs_idx =3D lua_gettop(L); + /* Push copy of arg we're processing */ lua_pushvalue(L, 1); - find_references(&dumper); + find_references_and_serialize(dumper.L, = dumper.anchortable_index, + dumper.serialized_objs_idx); =20 if (dump_root(&dumper) !=3D 0) goto out; diff --git a/src/box/lua/tuple.c b/src/box/lua/tuple.c index f7198a025..2779cf786 100644 --- a/src/box/lua/tuple.c +++ b/src/box/lua/tuple.c @@ -427,7 +427,7 @@ luamp_convert_key(struct lua_State *L, struct = luaL_serializer *cfg, return tuple_to_mpstream(tuple, stream); =20 struct luaL_field field; - if (luaL_tofield(L, cfg, NULL, index, &field) < 0) + if (luaL_tofield(L, cfg, 0, NULL, index, &field) < 0) luaT_error(L); if (field.type =3D=3D MP_ARRAY) { lua_pushvalue(L, index); diff --git a/src/box/sql/func.c b/src/box/sql/func.c index 9b6179f3a..2c7181f84 100644 --- a/src/box/sql/func.c +++ b/src/box/sql/func.c @@ -264,7 +264,7 @@ port_lua_get_vdbemem(struct port *base, uint32_t = *size) return NULL; for (int i =3D 0; i < argc; i++) { struct luaL_field field; - if (luaL_tofield(L, luaL_msgpack_default, + if (luaL_tofield(L, luaL_msgpack_default, 0, NULL, -1 - i, &field) < 0) { goto error; } diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c index 1e74a6a3c..f9503a275 100644 --- a/src/lua/msgpack.c +++ b/src/lua/msgpack.c @@ -157,11 +157,11 @@ restart: /* used by MP_EXT of unidentified subtype = */ lua_pushnil(L); /* first key */ while (lua_next(L, top) !=3D 0) { lua_pushvalue(L, -2); /* push a copy of key to = top */ - if (luaL_tofield(L, cfg, opts, lua_gettop(L), = field) < 0) + if (luaL_tofield(L, cfg, 0, opts, lua_gettop(L), = field) < 0) return luaT_error(L); luamp_encode_r(L, cfg, opts, stream, field, = level + 1); lua_pop(L, 1); /* pop a copy of key */ - if (luaL_tofield(L, cfg, opts, lua_gettop(L), = field) < 0) + if (luaL_tofield(L, cfg, 0, opts, lua_gettop(L), = field) < 0) return luaT_error(L); luamp_encode_r(L, cfg, opts, stream, field, = level + 1); lua_pop(L, 1); /* pop value */ @@ -182,7 +182,7 @@ restart: /* used by MP_EXT of unidentified subtype = */ mpstream_encode_array(stream, size); for (uint32_t i =3D 0; i < size; i++) { lua_rawgeti(L, top, i + 1); - if (luaL_tofield(L, cfg, opts, top + 1, field) < = 0) + if (luaL_tofield(L, cfg, 0, opts, top + 1, = field) < 0) return luaT_error(L); luamp_encode_r(L, cfg, opts, stream, field, = level + 1); lua_pop(L, 1); @@ -205,7 +205,7 @@ restart: /* used by MP_EXT of unidentified subtype = */ if (type !=3D MP_EXT) return type; /* Value has been packed by = the trigger */ /* Try to convert value to serializable type */ - luaL_convertfield(L, cfg, top, field); + luaL_convertfield(L, cfg, 0, top, field); /* handled by luaL_convertfield */ assert(field->type !=3D MP_EXT); assert(lua_gettop(L) =3D=3D top); @@ -230,7 +230,7 @@ luamp_encode(struct lua_State *L, struct = luaL_serializer *cfg, } =20 struct luaL_field field; - if (luaL_tofield(L, cfg, opts, lua_gettop(L), &field) < 0) + if (luaL_tofield(L, cfg, 0, opts, lua_gettop(L), &field) < 0) return luaT_error(L); enum mp_type top_type =3D luamp_encode_r(L, cfg, opts, stream, = &field, 0); =20 diff --git a/src/lua/pickle.c b/src/lua/pickle.c index 65208b5b3..86f0f77e8 100644 --- a/src/lua/pickle.c +++ b/src/lua/pickle.c @@ -80,7 +80,7 @@ lbox_pack(struct lua_State *L) if (i > nargs) luaL_error(L, "pickle.pack: argument count does = not match " "the format"); - luaL_checkfield(L, luaL_msgpack_default, i, &field); + luaL_checkfield(L, luaL_msgpack_default, 0, i, &field); switch (*format) { case 'B': case 'b': diff --git a/src/lua/utils.c b/src/lua/utils.c index b5a6ca5b7..0580f4094 100644 --- a/src/lua/utils.c +++ b/src/lua/utils.c @@ -441,7 +441,8 @@ lua_gettable_wrapper(lua_State *L) =20 static void lua_field_inspect_ucdata(struct lua_State *L, struct luaL_serializer = *cfg, - int idx, struct luaL_field *field) + int serialized_objs_idx, int idx, + struct luaL_field *field) { if (!cfg->encode_load_metatables) return; @@ -463,59 +464,280 @@ lua_field_inspect_ucdata(struct lua_State *L, = struct luaL_serializer *cfg, lua_pcall(L, 1, 1, 0); /* replace obj with the unpacked value */ lua_replace(L, idx); - if (luaL_tofield(L, cfg, NULL, idx, field) < 0) + if (luaL_tofield(L, cfg, serialized_objs_idx, NULL, idx, + field) < 0) luaT_error(L); } /* else ignore lua_gettable exceptions */ lua_settop(L, top); /* remove temporary objects */ } =20 +int +get_anchor(struct lua_State *L, int anchortable_index, + unsigned int *anchor_number, const char **anchor) +{ + lua_pushvalue(L, -1); + lua_rawget(L, anchortable_index); + if (lua_toboolean(L, -1) =3D=3D 0) { + /* This element is not referenced. */ + lua_pop(L, 1); + *anchor =3D NULL; + return 0; + } + + if (lua_isboolean(L, -1) !=3D 0) { + /* + * This element is referenced more than once but + * has not been named. + */ + char buf[32]; + snprintf(buf, sizeof(buf), "%u", (*anchor_number)++); + lua_pop(L, 1); + /* Generate a string anchor and push to table. */ + lua_pushvalue(L, -1); + lua_pushstring(L, buf); + *anchor =3D lua_tostring(L, -1); + lua_rawset(L, anchortable_index); + return GET_ANCHOR_NAMED_NOT; + } else { + /* This is an aliased element. */ + *anchor =3D lua_tostring(L, -1); + lua_pop(L, 1); + return GET_ANCHOR_ALIASED; + } + return GET_ANCHOR_NO_REFS; +} + +void +find_references_and_serialize(struct lua_State *L, int = anchortable_index, + int serialized_objs_idx) +{ + int type =3D lua_type(L, -1); + if (type !=3D LUA_TTABLE) + return; + int node_idx =3D lua_gettop(L); + + /* + * Check if the node is already serialized i.e. the record + * is already in the map of serialized objects. + */ + lua_pushvalue(L, node_idx); + lua_rawget(L, serialized_objs_idx); + bool srlzd =3D lua_isnil(L, -1) =3D=3D 0; + /* + * This flag indicates that the object is being serialized + * in the current function call. + */ + bool serialization =3D false; + + if (!srlzd && luaL_getmetafield(L, node_idx, LUAL_SERIALIZE) !=3D = 0 && + lua_isfunction(L, -1) !=3D 0) { + /* Delete nil after lua_rawget() above. */ + lua_replace(L, -2); + srlzd =3D true; + serialization =3D true; + /* + * Copy object itself and call serialize function + * for it. + */ + lua_pushvalue(L, node_idx); + if (lua_pcall(L, 1, 1, 0) !=3D 0) { + diag_set(LuajitError, lua_tostring(L, -1)); + luaT_error(L); + } + /* + * Add the result of serialization to the + * serialized_objs map. Key is node (node_idx), + * value is the result of serialization (now on + * the top of stack). + */ + lua_pushvalue(L, node_idx); + lua_pushvalue(L, -2); + lua_rawset(L, serialized_objs_idx); + } + + if (srlzd) { + if (lua_type(L, -1) =3D=3D LUA_TTABLE) { + /* + * Now we will process the new node - the + * result of serialization. It is + * necessary to take it into account in + * the anchor table. + */ + node_idx =3D lua_gettop(L); + } else { + lua_pop(L, 1); + return; + } + } else { + if (lua_isnil(L, -1) =3D=3D 0) + /* + * Pop an extra field thrown out + * by luaL_getmetafield(), it was + * not a function. + */ + lua_pop(L, 1); + /* Delete nil after lua_rawget() above. */ + lua_pop(L, 1); + } + /* + * Take an entry from the anchor table about the current + * node. + */ + lua_pushvalue(L, node_idx); + lua_rawget(L, anchortable_index); + int newval =3D -1; + if (lua_isnil(L, -1) =3D=3D 1) + /* + * The node has not yet been encountered, it is + * bypassed for the first time. + */ + newval =3D 0; + else if (lua_toboolean(L, -1) =3D=3D 0) + /* The node has already met once. */ + newval =3D 1; + lua_pop(L, 1); + if (newval !=3D -1) { + lua_pushvalue(L, node_idx); + lua_pushboolean(L, newval); + lua_rawset(L, anchortable_index); + } + if (srlzd && !serialization) { + /* + * The node has already been serialized not in the + * current call, so there is no point in going + * further in depth and checking the leaves. Pop + * the processed sterilization result. + */ + lua_pop(L, 1); + return; + } + if (newval) + /* The node has already met twice or more, so + * there is no point in going further in depth and + * checking the leaves. + */ + return; + + /* Recursively process other table values. */ + lua_pushnil(L); + while (lua_next(L, node_idx) !=3D 0) { + /* Find references on value. */ + find_references_and_serialize(L, anchortable_index, + serialized_objs_idx); + lua_pop(L, 1); + /* Find references on key. */ + find_references_and_serialize(L, anchortable_index, + serialized_objs_idx); + } + if (serialization) + /* + * The loop above processed the nodes inside the + * serialization result. Pop it, so as not to + * break the integrity of the loop of the previous + * call in recursion. + */ + lua_pop(L, 1); + + return; +} + +enum try_serialize_ret_code +{ + + TRY_SERIALIZE_ERROR =3D -1, + TRY_SERIALIZE_RES_TABLE_NOT =3D 0, + TRY_SERIALIZE_RES_FROM_MAP, + TRY_SERIALIZE_DEFAULT_TABLE, +}; + /** * Call __serialize method of a table object by index - * if the former exists. + * if the former exists or pull result of serialization (if + * exists) from table with index @a serialized_objs_idx if it + * isn't 0. * * If __serialize does not exist then function does nothing - * and the function returns 1; + * and the function returns TRY_SERIALIZE_DEFAULT_TABLE; * * If __serialize exists, is a function (which doesn't * raise any error) then a result of serialization - * replaces old value by the index and the function returns 0; + * replaces old value by the index and the function returns + * TRY_SERIALIZE_RES_TABLE_NOT or TRY_SERIALIZE_DEFAULT_TABLE; * * If the serialization is a hint string (like 'array' or 'map'), * then field->type, field->size and field->compact - * are set if necessary and the function returns 0; + * are set if necessary and the function returns + * TRY_SERIALIZE_RES_TABLE_NOT; * - * Otherwise it is an error, set diag and the funciton returns -1; + * Otherwise it is an error, set diag and the funciton returns + * TRY_SERIALIZE_ERROR; * * Return values: - * -1 - error occurs, diag is set, the top of guest stack - * is undefined. - * 0 - __serialize field is available in the metatable, - * the result value is put in the origin slot, - * encoding is finished. - * 1 - __serialize field is not available in the metatable, - * proceed with default table encoding. + * TRY_SERIALIZE_ERROR - error occurs, diag is set, the top of + * guest stack is undefined. + * TRY_SERIALIZE_RES_TABLE_NOT - __serialize field is available in + * the metatable, the result value is put in the origin slot, + * encoding is finished. + * TRY_SERIALIZE_RES_FROM_MAP - __serialize field is available in + * the metatable, the result value is table from serialized + * objects map. + * TRY_SERIALIZE_DEFAULT_TABLE - __serialize field is not + * available in the metatable, proceed with default table + * encoding. */ + static int -lua_field_try_serialize(struct lua_State *L, struct luaL_serializer = *cfg, - int idx, struct luaL_field *field) +lua_field_try_serialize(struct lua_State *L, int serialized_objs_idx, + struct luaL_serializer *cfg, int idx, + struct luaL_field *field) { if (luaL_getmetafield(L, idx, LUAL_SERIALIZE) =3D=3D 0) - return 1; + return TRY_SERIALIZE_DEFAULT_TABLE; if (lua_isfunction(L, -1)) { - /* copy object itself */ - lua_pushvalue(L, idx); - if (lua_pcall(L, 1, 1, 0) !=3D 0) { - diag_set(LuajitError, lua_tostring(L, -1)); - return -1; + if (serialized_objs_idx !=3D 0) { + /* + * Pop the __serialize function for the + * current node. It was already called in + * find_references_and_serialize(). + */ + lua_pop(L, 1); + /* + * Get the result of serialization from + * the map. + */ + lua_pushvalue(L, idx); + lua_rawget(L, serialized_objs_idx); + assert(lua_isnil(L, -1) =3D=3D 0); + + /* + * Replace the serialized node with a new + * result, if it is a table. + */ + if (lua_type(L, -1) =3D=3D LUA_TTABLE) { + lua_replace(L, idx); + return TRY_SERIALIZE_RES_FROM_MAP; + } + } else { + /* + * Serializer don't use map with + * serialized objects. Copy object itself + * and call __serialize for it. + */ + lua_pushvalue(L, idx); + if (lua_pcall(L, 1, 1, 0) !=3D 0) { + diag_set(LuajitError, lua_tostring(L, = -1)); + return TRY_SERIALIZE_ERROR; + } } - if (luaL_tofield(L, cfg, NULL, -1, field) !=3D 0) - return -1; + if (luaL_tofield(L, cfg, serialized_objs_idx, NULL, -1, + field) !=3D 0) + return TRY_SERIALIZE_ERROR; lua_replace(L, idx); - return 0; + return TRY_SERIALIZE_RES_TABLE_NOT; } if (!lua_isstring(L, -1)) { diag_set(LuajitError, "invalid " LUAL_SERIALIZE " = value"); - return -1; + return TRY_SERIALIZE_ERROR; } const char *type =3D lua_tostring(L, -1); if (strcmp(type, "array") =3D=3D 0 || strcmp(type, "seq") =3D=3D = 0 || @@ -533,16 +755,17 @@ lua_field_try_serialize(struct lua_State *L, = struct luaL_serializer *cfg, field->compact =3D true; } else { diag_set(LuajitError, "invalid " LUAL_SERIALIZE " = value"); - return -1; + return TRY_SERIALIZE_ERROR; } /* Remove value set by luaL_getmetafield. */ lua_pop(L, 1); - return 0; + return TRY_SERIALIZE_RES_TABLE_NOT; } =20 static int -lua_field_inspect_table(struct lua_State *L, struct luaL_serializer = *cfg, - int idx, struct luaL_field *field) +lua_field_inspect_table(struct lua_State *L, int serialized_objs_idx, + struct luaL_serializer *cfg, int idx, + struct luaL_field *field) { assert(lua_type(L, idx) =3D=3D LUA_TTABLE); uint32_t size =3D 0; @@ -550,14 +773,20 @@ lua_field_inspect_table(struct lua_State *L, = struct luaL_serializer *cfg, =20 if (cfg->encode_load_metatables) { int top =3D lua_gettop(L); - int res =3D lua_field_try_serialize(L, cfg, idx, field); - if (res =3D=3D -1) + int res =3D lua_field_try_serialize(L, = serialized_objs_idx, cfg, + idx, field); + if (res =3D=3D TRY_SERIALIZE_ERROR) return -1; assert(lua_gettop(L) =3D=3D top); (void)top; - if (res =3D=3D 0) + if (res =3D=3D TRY_SERIALIZE_RES_TABLE_NOT || + (res =3D=3D TRY_SERIALIZE_RES_FROM_MAP && + lua_type(L, -1) !=3D LUA_TTABLE)) return 0; - /* Fallthrough with res =3D=3D 1 */ + /* + * Fallthrough with res TRY_SERIALIZE_RES_FROM_MAP + * or TRY_SERIALIZE_DEFAULT_TABLE. + */ } =20 field->type =3D MP_ARRAY; @@ -612,14 +841,14 @@ lua_field_tostring(struct lua_State *L, struct = luaL_serializer *cfg, int idx, lua_call(L, 1, 1); lua_replace(L, idx); lua_settop(L, top); - if (luaL_tofield(L, cfg, NULL, idx, field) < 0) + if (luaL_tofield(L, cfg, 0, NULL, idx, field) < 0) luaT_error(L); } =20 int luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, - const struct serializer_opts *opts, int index, - struct luaL_field *field) + int serialized_objs_idx, const struct serializer_opts = *opts, + int index, struct luaL_field *field) { if (index < 0) index =3D lua_gettop(L) + index + 1; @@ -753,7 +982,8 @@ luaL_tofield(struct lua_State *L, struct = luaL_serializer *cfg, case LUA_TTABLE: { field->compact =3D false; - return lua_field_inspect_table(L, cfg, index, field); + return lua_field_inspect_table(L, serialized_objs_idx, = cfg, + index, field); } case LUA_TLIGHTUSERDATA: case LUA_TUSERDATA: @@ -773,8 +1003,8 @@ luaL_tofield(struct lua_State *L, struct = luaL_serializer *cfg, } =20 void -luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, int = idx, - struct luaL_field *field) +luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, + int serialized_objs_idx, int idx, struct luaL_field = *field) { if (idx < 0) idx =3D lua_gettop(L) + idx + 1; @@ -789,9 +1019,12 @@ luaL_convertfield(struct lua_State *L, struct = luaL_serializer *cfg, int idx, */ GCcdata *cd =3D cdataV(L->base + idx - 1); if (cd->ctypeid > CTID_CTYPEID) - lua_field_inspect_ucdata(L, cfg, idx, = field); + lua_field_inspect_ucdata(L, cfg, + = serialized_objs_idx, idx, + field); } else if (type =3D=3D LUA_TUSERDATA) { - lua_field_inspect_ucdata(L, cfg, idx, field); + lua_field_inspect_ucdata(L, cfg, = serialized_objs_idx, idx, + field); } } =20 diff --git a/src/lua/utils.h b/src/lua/utils.h index 4a164868b..cb1025495 100644 --- a/src/lua/utils.h +++ b/src/lua/utils.h @@ -353,6 +353,44 @@ struct luaL_field { bool compact; /* a flag used by YAML serializer = */ }; =20 +/** + * Find references to tables to look up self references in tables + * and serialize nodes if __serialize is function. Collect + * referenced nodes in a table with stack index @a + * anchortable_index and serialized nodes with results in a table + * with stack index @a serialized_objs_idx. + * + * @param L Lua stack. + * @param anchortable_index Index of anchor table on stack with + pairs: "node address" <-> "false (met once) or true + (met more than once)". + * @param serialized_objs_idx Index of serialized objects map on + * stack with pairs: "node address" <-> "result address". + */ +void +find_references_and_serialize(struct lua_State *L, int = anchortable_index, + int serialized_objs_idx); + +enum get_anchor_ret_code +{ + GET_ANCHOR_NO_REFS =3D 0, + GET_ANCHOR_NAMED_NOT, + GET_ANCHOR_ALIASED, +}; + +/** + * Generate aliases and anchor numbers for self references. + * + * @param L Lua stack. + * @param anchortable_index Index of anchor table on stack. + * @param[out] anchor_number Ptr to the number anchors in a + structure being under dumping. + * @param[out] anchor Ptr to anchor string. + */ +int +get_anchor(struct lua_State *L, int anchortable_index, + unsigned int *anchor_number, const char **anchor); + /** * @brief Convert a value from the Lua stack to a lua_field structure. * This function is designed for use with Lua bindings and data @@ -387,6 +425,8 @@ struct luaL_field { * * @param L stack * @param cfg configuration + * @param serialized_objs_idx Index of table with serialized + objects map. * @param opts the Lua serializer additional options. * @param index stack index * @param field conversion result @@ -396,8 +436,8 @@ struct luaL_field { */ int luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, - const struct serializer_opts *opts, int index, - struct luaL_field *field); + int serialized_objs_idx, const struct serializer_opts = *opts, + int index, struct luaL_field *field); =20 /** * @brief Try to convert userdata/cdata values using defined conversion = logic. @@ -405,18 +445,22 @@ luaL_tofield(struct lua_State *L, struct = luaL_serializer *cfg, * * @param L stack * @param cfg configuration + * @param serialized_objs_idx Index of table with serialized + objects map. * @param idx stack index * @param field conversion result */ void -luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, int = idx, - struct luaL_field *field); +luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, + int serialized_objs_idx, int idx, struct luaL_field = *field); =20 /** * @brief A wrapper for luaL_tofield() and luaL_convertfield() that * tries to convert value or raise an error. * @param L stack * @param cfg configuration + * @param serialized_objs_idx Stack index of table with serialized + objects map. * @param idx stack index * @param field conversion result * @sa lua_tofield() @@ -432,14 +476,14 @@ luaL_convertfield(struct lua_State *L, struct = luaL_serializer *cfg, int idx, * (tostring) -> (nil) -> exception */ static inline void -luaL_checkfield(struct lua_State *L, struct luaL_serializer *cfg, int = idx, - struct luaL_field *field) +luaL_checkfield(struct lua_State *L, struct luaL_serializer *cfg, + int serialized_objs_idx, int idx, struct luaL_field = *field) { - if (luaL_tofield(L, cfg, NULL, idx, field) < 0) + if (luaL_tofield(L, cfg, serialized_objs_idx, NULL, idx, field) = < 0) luaT_error(L); if (field->type !=3D MP_EXT || field->ext_type !=3D = MP_UNKNOWN_EXTENSION) return; - luaL_convertfield(L, cfg, idx, field); + luaL_convertfield(L, cfg, serialized_objs_idx, idx, field); } =20 void diff --git a/test/app/gh-3228-serializer-look-for-recursion.result = b/test/app/gh-3228-serializer-look-for-recursion.result new file mode 100644 index 000000000..c284fec6e --- /dev/null +++ b/test/app/gh-3228-serializer-look-for-recursion.result @@ -0,0 +1,164 @@ +-- test-run result file version 2 +test_run =3D require('test_run').new() + | --- + | ... + +-- +-- gh-3228: Check that recursive structures are serialized +-- properly. +-- +setmetatable({}, {__serialize =3D function(a) return a end}) + | --- + | - [] + | ... +setmetatable({}, {__serialize =3D function(a) return {a} end}) + | --- + | - &0 + | - *0 + | ... +setmetatable({}, {__serialize =3D function(a) return {a, a} end}) + | --- + | - &0 + | - *0 + | - *0 + | ... +setmetatable({}, {__serialize =3D function(a) return {a, a, a} end}) + | --- + | - &0 + | - *0 + | - *0 + | - *0 + | ... +setmetatable({}, {__serialize =3D function(a) return {{a, a}, a} end}) + | --- + | - &0 + | - - *0 + | - *0 + | - *0 + | ... +setmetatable({}, {__serialize =3D function(a) return {a, 1} end}) + | --- + | - &0 + | - *0 + | - 1 + | ... +setmetatable({}, {__serialize =3D function(a) return {{{{a}}}} end}) + | --- + | - &0 + | - - - - *0 + | ... + +b =3D {} + | --- + | ... +setmetatable({b}, {__serialize =3D function(a) return {a_1 =3D a, a_2 =3D= a, b_1 =3D b, b_2 =3D b} end}) + | --- + | - &0 + | b_2: &1 [] + | a_2: *0 + | a_1: *0 + | b_1: *1 + | ... +setmetatable({b}, {__serialize =3D function(a) return {a_1 =3D a, a_2 =3D= {a, b}, b =3D b} end}) + | --- + | - &0 + | b: &1 [] + | a_2: + | - *0 + | - *1 + | a_1: *0 + | ... + +a =3D {} + | --- + | ... +a[a] =3D a + | --- + | ... +recf =3D function(t) return setmetatable({}, {__serialize =3D recf}) = end + | --- + | ... +setmetatable(a, {__serialize =3D recf}) return a + | --- + | - [] + | ... + +-- +-- __serialize function is pure, i.e. always returns identical +-- value for identical argument. Otherwise, the behavior is +-- undefined. So that, we ignore the side effects and just use the +-- value after the first serialization. +-- +a =3D {} + | --- + | ... +b =3D {} + | --- + | ... +b[a] =3D a + | --- + | ... +show_a =3D true + | --- + | ... +test_run:cmd('setopt delimiter ";"') + | --- + | - true + | ... +serialize =3D function() + show_a =3D not show_a + if show_a then + return "a" + else + return "b" end +end; + | --- + | ... +test_run:cmd('setopt delimiter ""'); + | --- + | - true + | ... +setmetatable(a, {__serialize =3D serialize}) + | --- + | - b + | ... +b + | --- + | - a: a + | ... + +test_run:cmd('setopt delimiter ";"') + | --- + | - true + | ... +function example() + local a =3D {} + local b =3D {} + b[a] =3D a + local reta + local retb + local function swap_ser(o) + local newf + local f =3D getmetatable(o).__serialize + if f =3D=3D reta then + newf =3D retb + else + newf =3D reta + end + getmetatable(o).__serialize =3D newf + end + reta =3D function(o) swap_ser(o) return "a" end + retb =3D function(o) swap_ser(o) return "b" end + setmetatable(a, {__serialize =3D reta}) + return b +end; + | --- + | ... +test_run:cmd('setopt delimiter ""'); + | --- + | - true + | ... +example() + | --- + | - a: a + | ... diff --git a/test/app/gh-3228-serializer-look-for-recursion.test.lua = b/test/app/gh-3228-serializer-look-for-recursion.test.lua new file mode 100644 index 000000000..e1dbd9f90 --- /dev/null +++ b/test/app/gh-3228-serializer-look-for-recursion.test.lua @@ -0,0 +1,69 @@ +test_run =3D require('test_run').new() + +-- +-- gh-3228: Check that recursive structures are serialized +-- properly. +-- +setmetatable({}, {__serialize =3D function(a) return a end}) +setmetatable({}, {__serialize =3D function(a) return {a} end}) +setmetatable({}, {__serialize =3D function(a) return {a, a} end}) +setmetatable({}, {__serialize =3D function(a) return {a, a, a} end}) +setmetatable({}, {__serialize =3D function(a) return {{a, a}, a} end}) +setmetatable({}, {__serialize =3D function(a) return {a, 1} end}) +setmetatable({}, {__serialize =3D function(a) return {{{{a}}}} end}) + +b =3D {} +setmetatable({b}, {__serialize =3D function(a) return {a_1 =3D a, a_2 =3D= a, b_1 =3D b, b_2 =3D b} end}) +setmetatable({b}, {__serialize =3D function(a) return {a_1 =3D a, a_2 =3D= {a, b}, b =3D b} end}) + +a =3D {} +a[a] =3D a +recf =3D function(t) return setmetatable({}, {__serialize =3D recf}) = end +setmetatable(a, {__serialize =3D recf}) return a + +-- +-- __serialize function is pure, i.e. always returns identical +-- value for identical argument. Otherwise, the behavior is +-- undefined. So that, we ignore the side effects and just use the +-- value after the first serialization. +-- +a =3D {} +b =3D {} +b[a] =3D a +show_a =3D true +test_run:cmd('setopt delimiter ";"') +serialize =3D function() + show_a =3D not show_a + if show_a then + return "a" + else + return "b" end +end; +test_run:cmd('setopt delimiter ""'); +setmetatable(a, {__serialize =3D serialize}) +b + +test_run:cmd('setopt delimiter ";"') +function example() + local a =3D {} + local b =3D {} + b[a] =3D a + local reta + local retb + local function swap_ser(o) + local newf + local f =3D getmetatable(o).__serialize + if f =3D=3D reta then + newf =3D retb + else + newf =3D reta + end + getmetatable(o).__serialize =3D newf + end + reta =3D function(o) swap_ser(o) return "a" end + retb =3D function(o) swap_ser(o) return "b" end + setmetatable(a, {__serialize =3D reta}) + return b +end; +test_run:cmd('setopt delimiter ""'); +example() diff --git a/test/swim/swim.result b/test/swim/swim.result index bfc83c143..539131677 100644 --- a/test/swim/swim.result +++ b/test/swim/swim.result @@ -1322,16 +1322,13 @@ m_list incarnation: cdata {generation =3D 0ULL, version =3D 1ULL} uuid: 00000000-0000-1000-8000-000000000002 payload_size: 0 - - uri: 127.0.0.1: - status: alive - incarnation: cdata {generation =3D 0ULL, version =3D 2ULL} - uuid: 00000000-0000-1000-8000-000000000001 - payload_size: 8 - - uri: 127.0.0.1: + - &0 + uri: 127.0.0.1: status: alive incarnation: cdata {generation =3D 0ULL, version =3D 2ULL} uuid: 00000000-0000-1000-8000-000000000001 payload_size: 8 + - *0 ... e_list --- @@ -1374,16 +1371,13 @@ fiber.sleep(0) -- Two events - status update to 'left', and 'drop'. m_list --- -- - uri: 127.0.0.1: - status: left - incarnation: cdata {generation =3D 0ULL, version =3D 1ULL} - uuid: 00000000-0000-1000-8000-000000000002 - payload_size: 0 - - uri: 127.0.0.1: +- - &0 + uri: 127.0.0.1: status: left incarnation: cdata {generation =3D 0ULL, version =3D 1ULL} uuid: 00000000-0000-1000-8000-000000000002 payload_size: 0 + - *0 ... e_list --- diff --git a/third_party/lua-cjson/lua_cjson.c = b/third_party/lua-cjson/lua_cjson.c index 38e999870..67a9762bc 100644 --- a/third_party/lua-cjson/lua_cjson.c +++ b/third_party/lua-cjson/lua_cjson.c @@ -354,7 +354,7 @@ static void json_append_object(lua_State *l, struct = luaL_serializer *cfg, comma =3D 1; =20 struct luaL_field field; - luaL_checkfield(l, cfg, -2, &field); + luaL_checkfield(l, cfg, 0, -2, &field); if (field.type =3D=3D MP_UINT) { strbuf_append_char(json, '"'); json_append_uint(cfg, json, field.ival); @@ -384,7 +384,7 @@ static void json_append_data(lua_State *l, struct = luaL_serializer *cfg, int current_depth, strbuf_t *json) { struct luaL_field field; - luaL_checkfield(l, cfg, -1, &field); + luaL_checkfield(l, cfg, 0, -1, &field); switch (field.type) { case MP_UINT: return json_append_uint(cfg, json, field.ival); diff --git a/third_party/lua-yaml/lyaml.cc = b/third_party/lua-yaml/lyaml.cc index 9c3a4a646..249603ba3 100644 --- a/third_party/lua-yaml/lyaml.cc +++ b/third_party/lua-yaml/lyaml.cc @@ -79,6 +79,7 @@ struct lua_yaml_dumper { lua_State *L; struct luaL_serializer *cfg; int anchortable_index; + int serialized_objs_idx; unsigned int anchor_number; yaml_emitter_t emitter; char error; @@ -500,33 +501,16 @@ usage_error: static int dump_node(struct lua_yaml_dumper *dumper); =20 static yaml_char_t *get_yaml_anchor(struct lua_yaml_dumper *dumper) { - const char *s =3D ""; - lua_pushvalue(dumper->L, -1); - lua_rawget(dumper->L, dumper->anchortable_index); - if (!lua_toboolean(dumper->L, -1)) { - lua_pop(dumper->L, 1); - return NULL; - } - - if (lua_isboolean(dumper->L, -1)) { - /* this element is referenced more than once but has not been = named */ - char buf[32]; - snprintf(buf, sizeof(buf), "%u", dumper->anchor_number++); - lua_pop(dumper->L, 1); - lua_pushvalue(dumper->L, -1); - lua_pushstring(dumper->L, buf); - s =3D lua_tostring(dumper->L, -1); - lua_rawset(dumper->L, dumper->anchortable_index); - } else { - /* this is an aliased element */ + const char *anchor =3D NULL; + if (get_anchor(dumper->L, dumper->anchortable_index, = &dumper->anchor_number, + &anchor) =3D=3D GET_ANCHOR_ALIASED) { yaml_event_t ev; - const char *str =3D lua_tostring(dumper->L, -1); - if (!yaml_alias_event_initialize(&ev, (yaml_char_t *) str) || + if (!yaml_alias_event_initialize(&ev, (yaml_char_t *) anchor) || !yaml_emitter_emit(&dumper->emitter, &ev)) luaL_error(dumper->L, OOM_ERRMSG); - lua_pop(dumper->L, 1); + return (yaml_char_t *)""; } - return (yaml_char_t *)s; + return (yaml_char_t *)anchor; } =20 static int dump_table(struct lua_yaml_dumper *dumper, struct luaL_field = *field){ @@ -622,7 +606,8 @@ static int dump_node(struct lua_yaml_dumper *dumper) (void) unused; =20 int top =3D lua_gettop(dumper->L); - luaL_checkfield(dumper->L, dumper->cfg, top, &field); + luaL_checkfield(dumper->L, dumper->cfg, dumper->serialized_objs_idx, + top, &field); switch(field.type) { case MP_UINT: snprintf(buf, sizeof(buf) - 1, "%" PRIu64, field.ival); @@ -745,35 +730,6 @@ static int append_output(void *arg, unsigned char = *buf, size_t len) { return 1; } =20 -static void find_references(struct lua_yaml_dumper *dumper) { - int newval =3D -1, type =3D lua_type(dumper->L, -1); - if (type !=3D LUA_TTABLE) - return; - - lua_pushvalue(dumper->L, -1); /* push copy of table */ - lua_rawget(dumper->L, dumper->anchortable_index); - if (lua_isnil(dumper->L, -1)) - newval =3D 0; - else if (!lua_toboolean(dumper->L, -1)) - newval =3D 1; - lua_pop(dumper->L, 1); - if (newval !=3D -1) { - lua_pushvalue(dumper->L, -1); - lua_pushboolean(dumper->L, newval); - lua_rawset(dumper->L, dumper->anchortable_index); - } - if (newval) - return; - - /* recursively process other table values */ - lua_pushnil(dumper->L); - while (lua_next(dumper->L, -2) !=3D 0) { - find_references(dumper); /* find references on value */ - lua_pop(dumper->L, 1); - find_references(dumper); /* find references on key */ - } -} - int lua_yaml_encode(lua_State *L, struct luaL_serializer *serializer, const char *tag_handle, const char *tag_prefix) @@ -817,9 +773,12 @@ lua_yaml_encode(lua_State *L, struct = luaL_serializer *serializer, =20 lua_newtable(L); dumper.anchortable_index =3D lua_gettop(L); + lua_newtable(L); + dumper.serialized_objs_idx =3D lua_gettop(L); dumper.anchor_number =3D 0; lua_pushvalue(L, 1); /* push copy of arg we're processing */ - find_references(&dumper); + find_references_and_serialize(dumper.L, dumper.anchortable_index, + dumper.serialized_objs_idx); dump_document(&dumper); if (dumper.error) goto error;