From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp46.i.mail.ru (smtp46.i.mail.ru [94.100.177.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 5607F4696C4 for ; Fri, 10 Apr 2020 02:51:01 +0300 (MSK) From: Serge Petrenko Date: Fri, 10 Apr 2020 02:50:41 +0300 Message-Id: <60281b83d48279b708ad173d271e900943ce8e57.1586476073.git.sergepetrenko@tarantool.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH v2 3/4] box: add MsgPack encoding/decoding for UUID List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: v.shpilevoy@tarantool.org Cc: tarantool-patches@dev.tarantool.org A special format for encoding UUIDs to MsgPack is introduced. It is supported by both lua and C encoders/decoders, and it is now possible to insert UUIDs into spaces, but only into unindexed fields without format for now. Prerequisite #4268 @TarantoolBot document Title: Internals: msgpack format for UUID UUID values share the MessagePack type with decimals: both use MP_EXT. A new subtype is introduced for UUIDs, MP_UUID = `0x02` UUID is encoded as follows: ``` +--------+---------+-----------+ | MP_EXT | MP_UUID | UuidValue | +--------+---------+-----------+ ``` Since UUID is 16 bytes in size, the header, MP_EXT, is always the same: `0xd8`. MP_UUID = `0x02` follows. The header is followed by the 16 bytes of the UuidValue. UuidValue consists of 11 fields, which are encoded as big endian unsigned integers in the following order: `time_low` (4 bytes), `time_mid` (2 bytes), `time_hi_and_version` (2 bytes), `clock_seq_hi_and_reserved` (1 byte), `clock_seq_low` (1 byte), `node[0], ..., node[5]` (1 byte each). The total size of such a representation is 18 bytes, whereas storing uuids as strings requires from 34 (when '-'s are ommitted) to 38 bytes per UUID, giving a 2x space usage improvement. --- extra/exports | 3 + src/lib/core/mp_extension_types.h | 2 + src/lib/mpstream/mpstream.c | 11 ++++ src/lib/mpstream/mpstream.h | 5 ++ src/lib/uuid/CMakeLists.txt | 2 +- src/lib/uuid/mp_uuid.c | 98 ++++++++++++++++++++++++++++ src/lib/uuid/mp_uuid.h | 90 +++++++++++++++++++++++++ src/lua/msgpack.c | 27 ++++++-- src/lua/msgpackffi.lua | 14 ++++ src/lua/utils.c | 20 ++++++ src/lua/utils.h | 5 ++ src/lua/uuid.lua | 9 --- test/app-tap/lua/serializer_test.lua | 8 +++ test/app-tap/msgpackffi.test.lua | 3 +- test/app/msgpack.result | 21 ++++++ test/app/msgpack.test.lua | 13 ++++ test/app/uuid.result | 13 +++- test/app/uuid.test.lua | 5 ++ test/box/tuple.result | 81 +++++++++++++++++++++++ test/box/tuple.test.lua | 25 +++++++ test/unit/uuid.c | 25 ++++++- test/unit/uuid.result | 8 ++- third_party/lua-cjson/lua_cjson.c | 27 +++++--- third_party/lua-yaml/lyaml.cc | 17 ++++- 24 files changed, 500 insertions(+), 32 deletions(-) create mode 100644 src/lib/uuid/mp_uuid.c create mode 100644 src/lib/uuid/mp_uuid.h diff --git a/extra/exports b/extra/exports index 9323996c1..1a20cc69f 100644 --- a/extra/exports +++ b/extra/exports @@ -63,11 +63,14 @@ crc32_calc mp_encode_double mp_encode_float mp_encode_decimal +mp_encode_uuid mp_decode_double mp_decode_float mp_decode_extl mp_sizeof_decimal +mp_sizeof_uuid decimal_unpack +uuid_unpack log_type say_set_log_level diff --git a/src/lib/core/mp_extension_types.h b/src/lib/core/mp_extension_types.h index bc9873f68..7d42f212b 100644 --- a/src/lib/core/mp_extension_types.h +++ b/src/lib/core/mp_extension_types.h @@ -42,6 +42,8 @@ enum mp_extension_type { MP_UNKNOWN_EXTENSION = 0, MP_DECIMAL = 1, + MP_UUID = 2, + mp_extension_type_MAX, }; #endif diff --git a/src/lib/mpstream/mpstream.c b/src/lib/mpstream/mpstream.c index a737212c1..01b2925f5 100644 --- a/src/lib/mpstream/mpstream.c +++ b/src/lib/mpstream/mpstream.c @@ -34,6 +34,7 @@ #include #include "msgpuck.h" #include "core/mp_decimal.h" +#include "uuid/mp_uuid.h" void mpstream_reserve_slow(struct mpstream *stream, size_t size) @@ -197,6 +198,16 @@ mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val) mpstream_advance(stream, pos - data); } +void +mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid) +{ + char *data = mpstream_reserve(stream, mp_sizeof_uuid()); + if (data == NULL) + return; + char *pos = mp_encode_uuid(data, uuid); + mpstream_advance(stream, pos - data); +} + void mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n) { diff --git a/src/lib/mpstream/mpstream.h b/src/lib/mpstream/mpstream.h index 3a022daa0..a60add143 100644 --- a/src/lib/mpstream/mpstream.h +++ b/src/lib/mpstream/mpstream.h @@ -38,6 +38,8 @@ extern "C" { #endif /* defined(__cplusplus) */ +struct tt_uuid; + /** * Ask the allocator to reserve at least size bytes. It can reserve * more, and update *size with the new size. @@ -140,6 +142,9 @@ mpstream_encode_binl(struct mpstream *stream, uint32_t len); void mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val); +void +mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid); + /** Copies n bytes from memory area src to stream. */ void mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n); diff --git a/src/lib/uuid/CMakeLists.txt b/src/lib/uuid/CMakeLists.txt index 581e4495a..a3316647b 100644 --- a/src/lib/uuid/CMakeLists.txt +++ b/src/lib/uuid/CMakeLists.txt @@ -1,2 +1,2 @@ -add_library(uuid STATIC tt_uuid.c) +add_library(uuid STATIC tt_uuid.c mp_uuid.c) target_link_libraries(uuid core bit) diff --git a/src/lib/uuid/mp_uuid.c b/src/lib/uuid/mp_uuid.c new file mode 100644 index 000000000..7acfbc797 --- /dev/null +++ b/src/lib/uuid/mp_uuid.c @@ -0,0 +1,98 @@ +/* + * Copyright 2020, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include "mp_uuid.h" +#include "msgpuck.h" +#include "mp_extension_types.h" +#include "lib/uuid/tt_uuid.h" + +inline uint32_t +mp_sizeof_uuid(void) +{ + return mp_sizeof_ext(UUID_PACKED_LEN); +} + +char * +uuid_pack(char *data, const struct tt_uuid *uuid) +{ + data = mp_store_u32(data, uuid->time_low); + data = mp_store_u16(data, uuid->time_mid); + data = mp_store_u16(data, uuid->time_hi_and_version); + data = mp_store_u8(data, uuid->clock_seq_hi_and_reserved); + data = mp_store_u8(data, uuid->clock_seq_low); + for (int i = 0; i < 6; i++) + data = mp_store_u8(data, uuid->node[i]); + return data; +} + +struct tt_uuid * +uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid) +{ + const char *const svp = *data; + if (len != UUID_PACKED_LEN) + return NULL; + uuid->time_low = mp_load_u32(data); + uuid->time_mid = mp_load_u16(data); + uuid->time_hi_and_version = mp_load_u16(data); + uuid->clock_seq_hi_and_reserved = mp_load_u8(data); + uuid->clock_seq_low = mp_load_u8(data); + for (int i = 0; i < 6; i++) + uuid->node[i] = mp_load_u8(data); + + if (tt_uuid_validate(uuid) != 0) { + *data = svp; + return NULL; + } + return uuid; +} + +struct tt_uuid * +mp_decode_uuid(const char **data, struct tt_uuid *uuid) +{ + if (mp_typeof(**data) != MP_EXT) + return NULL; + int8_t type; + const char *const svp = *data; + + uint32_t len = mp_decode_extl(data, &type); + if (type != MP_UUID || uuid_unpack(data, len, uuid) == NULL) { + *data = svp; + return NULL; + } + return uuid; +} + +char * +mp_encode_uuid(char *data, const struct tt_uuid *uuid) +{ + data = mp_encode_extl(data, MP_UUID, UUID_PACKED_LEN); + return uuid_pack(data, uuid); +} diff --git a/src/lib/uuid/mp_uuid.h b/src/lib/uuid/mp_uuid.h new file mode 100644 index 000000000..430cb96d2 --- /dev/null +++ b/src/lib/uuid/mp_uuid.h @@ -0,0 +1,90 @@ +#pragma once + +/* + * Copyright 2020, Tarantool AUTHORS, please see AUTHORS file. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * 1. Redistributions of source code must retain the above + * copyright notice, this list of conditions and the + * following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include + +#if defined(__cplusplus) +extern "C" { +#endif /* defined(__cplusplus) */ + +struct tt_uuid; + +#define UUID_PACKED_LEN sizeof(struct tt_uuid) + +/** + * \brief Return the number of bytes an encoded uuid value takes. + */ +uint32_t +mp_sizeof_uuid(void); + +/** + * Load a uuid value from the buffer. Can be used in a combination + * with mp_decode_extl() instead of mp_decode_uuid() when multiple + * extension types are possible. + * + * \param data A buffer. + * \param len Length returned by mp_decode_extl, has to be equal + * to sizeof(struct tt_uuid), otherwise an error is + * returned. + * \param[out] uuid Uuid to be decoded. + * \return A pointer to the decoded uuid. + * NULL in case of an error. + * \post *data = *data + sizeof(struct tt_uuid). + */ +struct tt_uuid * +uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid); + +/** + * \brief Decode a uuid from MsgPack \a data. + * \param data A buffer. + * \param[out] uuid Uuid to be decoded. + * \return A pointer to the decoded uuid. + * NULL in case of an error. + * \post *data = *data + mp_sizeof_uuid(). + */ +struct tt_uuid * +mp_decode_uuid(const char **data, struct tt_uuid *uuid); + +/** + * \brief Encode a uuid. + * \param data A buffer. + * \param uuid A uuid to encode. + * + * \return \a data + mp_sizeof_uuid() + */ +char * +mp_encode_uuid(char *data, const struct tt_uuid *uuid); + +#if defined(__cplusplus) +} /* extern "C" */ +#endif /* defined(__cplusplus) */ diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c index 73ed3ece6..e4fb0cf43 100644 --- a/src/lua/msgpack.c +++ b/src/lua/msgpack.c @@ -43,6 +43,7 @@ #include "lua/decimal.h" /* lua_pushdecimal() */ #include "lib/core/decimal.h" /* decimal_unpack() */ +#include "lib/uuid/mp_uuid.h" /* mp_decode_uuid() */ #include "lib/core/mp_extension_types.h" #include @@ -114,7 +115,7 @@ luamp_encode_r(struct lua_State *L, struct luaL_serializer *cfg, int top = lua_gettop(L); enum mp_type type; -restart: /* used by MP_EXT */ +restart: /* used by MP_EXT of unidentified subtype */ switch (field->type) { case MP_UINT: mpstream_encode_uint(stream, field->ival); @@ -190,7 +191,10 @@ restart: /* used by MP_EXT */ switch (field->ext_type) { case MP_DECIMAL: mpstream_encode_decimal(stream, field->decval); - return MP_EXT; + break; + case MP_UUID: + mpstream_encode_uuid(stream, field->uuidval); + break; default: /* Run trigger if type can't be encoded */ type = luamp_encode_extension(L, top, stream); @@ -310,10 +314,17 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg, { decimal_t *dec = lua_pushdecimal(L); dec = decimal_unpack(data, len, dec); - if (dec == NULL) { - lua_pop(L, -1); - luaL_error(L, "msgpack.decode: invalid MsgPack"); - } + if (dec == NULL) + goto ext_decode_err; + return; + } + case MP_UUID: + { + struct tt_uuid *uuid = luaL_pushuuid(L); + *data = svp; + uuid = mp_decode_uuid(data, uuid); + if (uuid == NULL) + goto ext_decode_err; return; } default: @@ -325,6 +336,10 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg, break; } } + return; +ext_decode_err: + lua_pop(L, -1); + luaL_error(L, "msgpack.decode: invalid MsgPack"); } diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua index f775f2d41..f01ffaef0 100644 --- a/src/lua/msgpackffi.lua +++ b/src/lua/msgpackffi.lua @@ -20,6 +20,10 @@ char * mp_encode_decimal(char *data, decimal_t *dec); uint32_t mp_sizeof_decimal(const decimal_t *dec); +char * +mp_encode_uuid(char *data, const struct tt_uuid *uuid); +uint32_t +mp_sizeof_uuid(); float mp_decode_float(const char **data); double @@ -28,6 +32,8 @@ uint32_t mp_decode_extl(const char **data, int8_t *type); decimal_t * decimal_unpack(const char **data, uint32_t len, decimal_t *dec); +struct tt_uuid * +uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid); ]]) local strict_alignment = (jit.arch == 'arm') @@ -129,6 +135,11 @@ local function encode_decimal(buf, num) builtin.mp_encode_decimal(p, num) end +local function encode_uuid(buf, uuid) + local p = buf:alloc(builtin.mp_sizeof_uuid()) + builtin.mp_encode_uuid(p, uuid) +end + local function encode_int(buf, num) if num >= 0 then if num <= 0x7f then @@ -311,6 +322,7 @@ on_encode(ffi.typeof('bool'), encode_bool_cdata) on_encode(ffi.typeof('float'), encode_float) on_encode(ffi.typeof('double'), encode_double) on_encode(ffi.typeof('decimal_t'), encode_decimal) +on_encode(ffi.typeof('struct tt_uuid'), encode_uuid) -------------------------------------------------------------------------------- -- Decoder @@ -495,6 +507,8 @@ local ext_decoder = { [0] = function(data, len) error("unsupported extension type") end, -- MP_DECIMAL [1] = function(data, len) local num = ffi.new("decimal_t") builtin.decimal_unpack(data, len, num) return num end, + -- MP_UUID + [2] = function(data, len) local uuid = ffi.new("struct tt_uuid") builtin.uuid_unpack(data, len, uuid) return uuid end, } local function decode_ext(data) diff --git a/src/lua/utils.c b/src/lua/utils.c index 54d18ac89..bd6bfb008 100644 --- a/src/lua/utils.c +++ b/src/lua/utils.c @@ -45,6 +45,7 @@ static uint32_t CTID_STRUCT_IBUF; static uint32_t CTID_STRUCT_IBUF_PTR; static uint32_t CTID_CHAR_PTR; static uint32_t CTID_CONST_CHAR_PTR; +static uint32_t CTID_UUID; uint32_t CTID_DECIMAL; @@ -101,6 +102,12 @@ luaL_pushcdata(struct lua_State *L, uint32_t ctypeid) return cdataptr(cd); } +struct tt_uuid * +luaL_pushuuid(struct lua_State *L) +{ + return luaL_pushcdata(L, CTID_UUID); +} + void * luaL_checkcdata(struct lua_State *L, int idx, uint32_t *ctypeid) { @@ -746,6 +753,9 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index, if (cd->ctypeid == CTID_DECIMAL) { field->ext_type = MP_DECIMAL; field->decval = (decimal_t *) cdata; + } else if (cd->ctypeid == CTID_UUID) { + field->ext_type = MP_UUID; + field->uuidval = (struct tt_uuid *) cdata; } else { field->ext_type = MP_UNKNOWN_EXTENSION; } @@ -1286,5 +1296,15 @@ tarantool_lua_utils_init(struct lua_State *L) assert(CTID_CHAR_PTR != 0); CTID_CONST_CHAR_PTR = luaL_ctypeid(L, "const char *"); assert(CTID_CONST_CHAR_PTR != 0); + rc = luaL_cdef(L, "struct tt_uuid {" + "uint32_t time_low;" + "uint16_t time_mid;" + "uint16_t time_hi_and_version;" + "uint8_t clock_seq_hi_and_reserved;" + "uint8_t clock_seq_low;" + "uint8_t node[6];" + "};"); + CTID_UUID = luaL_ctypeid(L, "struct tt_uuid"); + assert(CTID_UUID != 0); return 0; } diff --git a/src/lua/utils.h b/src/lua/utils.h index 0b3672769..4bc041796 100644 --- a/src/lua/utils.h +++ b/src/lua/utils.h @@ -61,6 +61,7 @@ extern "C" { struct lua_State; struct ibuf; +struct tt_uuid; /** * Single global lua_State shared by core and modules. @@ -71,6 +72,9 @@ struct ibuf; extern struct lua_State *tarantool_L; extern struct ibuf *tarantool_lua_ibuf; +struct tt_uuid * +luaL_pushuuid(struct lua_State *L); + /** \cond public */ /** @@ -320,6 +324,7 @@ struct luaL_field { /* Array or map. */ uint32_t size; decimal_t *decval; + struct tt_uuid *uuidval; }; enum mp_type type; /* subtypes of MP_EXT */ diff --git a/src/lua/uuid.lua b/src/lua/uuid.lua index f8418cf4d..5d19a4408 100644 --- a/src/lua/uuid.lua +++ b/src/lua/uuid.lua @@ -5,15 +5,6 @@ local static_alloc = require('buffer').static_alloc local builtin = ffi.C ffi.cdef[[ -struct tt_uuid { - uint32_t time_low; - uint16_t time_mid; - uint16_t time_hi_and_version; - uint8_t clock_seq_hi_and_reserved; - uint8_t clock_seq_low; - uint8_t node[6]; -}; - void tt_uuid_create(struct tt_uuid *uu); int diff --git a/test/app-tap/lua/serializer_test.lua b/test/app-tap/lua/serializer_test.lua index 47edac621..2a668f898 100644 --- a/test/app-tap/lua/serializer_test.lua +++ b/test/app-tap/lua/serializer_test.lua @@ -204,6 +204,13 @@ local function test_decimal(test, s) rt(test, s, decimal.new('-1234567891234567890.0987654321987654321'), 'cdata') end +local function test_uuid(test, s) + local uuid = require('uuid') + test:plan(2) + + rt(test, s, uuid.new(), 'cdata') +end + local function test_boolean(test, s) test:plan(4) @@ -505,6 +512,7 @@ return { test_table = test_table; test_ucdata = test_ucdata; test_decimal = test_decimal; + test_uuid = test_uuid; test_depth = test_depth; test_decode_buffer = test_decode_buffer; } diff --git a/test/app-tap/msgpackffi.test.lua b/test/app-tap/msgpackffi.test.lua index 994402d15..0ee5f5edc 100755 --- a/test/app-tap/msgpackffi.test.lua +++ b/test/app-tap/msgpackffi.test.lua @@ -118,11 +118,12 @@ end tap.test("msgpackffi", function(test) local serializer = require('msgpackffi') - test:plan(11) + test:plan(12) test:test("unsigned", common.test_unsigned, serializer) test:test("signed", common.test_signed, serializer) test:test("double", common.test_double, serializer) test:test("decimal", common.test_decimal, serializer) + test:test("uuid", common.test_uuid, serializer) test:test("boolean", common.test_boolean, serializer) test:test("string", common.test_string, serializer) test:test("nil", common.test_nil, serializer) diff --git a/test/app/msgpack.result b/test/app/msgpack.result index 4b5aec784..ddf06fc9d 100644 --- a/test/app/msgpack.result +++ b/test/app/msgpack.result @@ -293,3 +293,24 @@ msgpack.decode(msgpack.encode(e)) == e --- - true ... +-- +-- gh-4268: msgpack encode/decode UUID +-- +uuid = require('uuid') +--- +... +fail = nil +--- +... +for i = 1,10 do\ + local a = uuid.new()\ + if msgpack.decode(msgpack.encode(a)) ~= a then\ + fail = a\ + end\ +end +--- +... +fail +--- +- null +... diff --git a/test/app/msgpack.test.lua b/test/app/msgpack.test.lua index 9224d870a..17e05df5c 100644 --- a/test/app/msgpack.test.lua +++ b/test/app/msgpack.test.lua @@ -99,3 +99,16 @@ msgpack.decode(msgpack.encode(b)) == b msgpack.decode(msgpack.encode(c)) == c msgpack.decode(msgpack.encode(d)) == d msgpack.decode(msgpack.encode(e)) == e + +-- +-- gh-4268: msgpack encode/decode UUID +-- +uuid = require('uuid') +fail = nil +for i = 1,10 do\ + local a = uuid.new()\ + if msgpack.decode(msgpack.encode(a)) ~= a then\ + fail = a\ + end\ +end +fail diff --git a/test/app/uuid.result b/test/app/uuid.result index 0713614c6..751865e5f 100644 --- a/test/app/uuid.result +++ b/test/app/uuid.result @@ -2,6 +2,13 @@ uuid = require('uuid') --- ... +test_run = require('test_run').new() +--- +... +test_run:cmd("push filter ".."'\\.lua.*:[0-9]+: ' to '.lua '") +--- +- true +... -- -- RFC4122 compliance -- @@ -106,7 +113,7 @@ uu.node[5] -- invalid values uuid.fromstr(nil) --- -- error: 'builtin/uuid.lua:47: fromstr(str)' +- error: 'builtin/uuid.lua fromstr(str)' ... uuid.fromstr('') --- @@ -268,3 +275,7 @@ uu = nil uuid = nil --- ... +test_run:cmd("clear filter") +--- +- true +... diff --git a/test/app/uuid.test.lua b/test/app/uuid.test.lua index ecb0cc3fe..ac125cddc 100644 --- a/test/app/uuid.test.lua +++ b/test/app/uuid.test.lua @@ -1,6 +1,9 @@ -- box.uuid uuid = require('uuid') +test_run = require('test_run').new() +test_run:cmd("push filter ".."'\\.lua.*:[0-9]+: ' to '.lua '") + -- -- RFC4122 compliance -- @@ -96,3 +99,5 @@ uu.str() uu = nil uuid = nil + +test_run:cmd("clear filter") diff --git a/test/box/tuple.result b/test/box/tuple.result index a499aa43a..eb60a5645 100644 --- a/test/box/tuple.result +++ b/test/box/tuple.result @@ -1490,6 +1490,87 @@ box.tuple.is(box.tuple.new({1})) --- - true ... +-- +-- gh-4268 UUID in tuple +-- +uuid = require("uuid") +--- +... +-- Fixed randomly generated uuids to avoid problems with test +-- output comparison. +a = uuid.fromstr("c8f0fa1f-da29-438c-a040-393f1126ad39") +--- +... +b = uuid.fromstr("83eb4959-3de6-49fb-8890-6fb4423dd186") +--- +... +t = box.tuple.new(a, 2, b, "string") +--- +... +state, val = t:next() +--- +... +state +--- +- 1 +... +val == a +--- +- true +... +state, val = t:next(state) +--- +... +state +--- +- 2 +... +val +--- +- 2 +... +state, val = t:next(state) +--- +... +state +--- +- 3 +... +val == b +--- +- true +... +t:slice(1) +--- +- 2 +- 83eb4959-3de6-49fb-8890-6fb4423dd186 +- string +... +t:slice(-1) +--- +- string +... +t:slice(-2) +--- +- 83eb4959-3de6-49fb-8890-6fb4423dd186 +- string +... +msgpack.decode(msgpack.encode(t)) +--- +- [c8f0fa1f-da29-438c-a040-393f1126ad39, 2, 83eb4959-3de6-49fb-8890-6fb4423dd186, + 'string'] +- 46 +... +msgpackffi.decode(msgpackffi.encode(t)) +--- +- [c8f0fa1f-da29-438c-a040-393f1126ad39, 2, 83eb4959-3de6-49fb-8890-6fb4423dd186, + 'string'] +- 46 +... +t:bsize() +--- +- 45 +... msgpack.cfg({encode_max_depth = max_depth, encode_deep_as_nil = deep_as_nil}) --- ... diff --git a/test/box/tuple.test.lua b/test/box/tuple.test.lua index b83fca5cd..4201e9860 100644 --- a/test/box/tuple.test.lua +++ b/test/box/tuple.test.lua @@ -510,4 +510,29 @@ box.tuple.is('1') box.tuple.is(box.tuple.new()) box.tuple.is(box.tuple.new({1})) +-- +-- gh-4268 UUID in tuple +-- +uuid = require("uuid") +-- Fixed randomly generated uuids to avoid problems with test +-- output comparison. +a = uuid.fromstr("c8f0fa1f-da29-438c-a040-393f1126ad39") +b = uuid.fromstr("83eb4959-3de6-49fb-8890-6fb4423dd186") +t = box.tuple.new(a, 2, b, "string") +state, val = t:next() +state +val == a +state, val = t:next(state) +state +val +state, val = t:next(state) +state +val == b +t:slice(1) +t:slice(-1) +t:slice(-2) +msgpack.decode(msgpack.encode(t)) +msgpackffi.decode(msgpackffi.encode(t)) +t:bsize() + msgpack.cfg({encode_max_depth = max_depth, encode_deep_as_nil = deep_as_nil}) diff --git a/test/unit/uuid.c b/test/unit/uuid.c index c43d93b4f..b51d13cb8 100644 --- a/test/unit/uuid.c +++ b/test/unit/uuid.c @@ -1,5 +1,7 @@ #include "unit.h" #include "uuid/tt_uuid.h" +#include "uuid/mp_uuid.h" +#include "core/random.h" #include static void @@ -27,10 +29,29 @@ uuid_test(struct tt_uuid a, struct tt_uuid b, int expected_result) "%s %s %s", a_str, sign, b_str); } +static void +mp_uuid_test() +{ + plan(4); + char buf[18]; + char *data = buf; + const char *data1 = buf; + struct tt_uuid uu, ret; + random_init(); + tt_uuid_create(&uu); + char *end = mp_encode_uuid(data, &uu); + is(end - data, mp_sizeof_uuid(), "mp_sizeof_uuid() == encoded length"); + struct tt_uuid *rc = mp_decode_uuid(&data1, &ret); + is(rc, &ret, "mp_decode_uuid() return code"); + is(data1, end, "mp_sizeof_uuid() == decoded length"); + is(tt_uuid_compare(&uu, &ret), 0, "mp_decode_uuid(mp_encode_uuid(uu)) == uu"); + check_plan(); +} + int main(void) { - plan(2); + plan(3); uuid_test( (struct tt_uuid){.time_low = 1712399963, @@ -63,5 +84,7 @@ main(void) .node = "v\025Oo9I"}, -1); + mp_uuid_test(); + return check_plan(); } diff --git a/test/unit/uuid.result b/test/unit/uuid.result index 40ccce759..50a1140c5 100644 --- a/test/unit/uuid.result +++ b/test/unit/uuid.result @@ -1,3 +1,9 @@ -1..2 +1..3 ok 1 - 6611265b-8852-4832-af8b-4164d52c62eb > 186ebbf7-cf97-4e2e-8b1b-76154f6f3949 ok 2 - 075b4148-8fb0-2e7f-af50-4164d52c62eb < 1fbc929f-5da8-28c5-8b36-76154f6f3949 + 1..4 + ok 1 - mp_sizeof_uuid() == encoded length + ok 2 - mp_decode_uuid() return code + ok 3 - mp_sizeof_uuid() == decoded length + ok 4 - mp_decode_uuid(mp_encode_uuid(uu)) == uu +ok 3 - subtests diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c index 26e566a6f..6e1793a59 100644 --- a/third_party/lua-cjson/lua_cjson.c +++ b/third_party/lua-cjson/lua_cjson.c @@ -48,6 +48,9 @@ #include "strbuf.h" #include "lua/utils.h" +#include "lib/core/mp_extension_types.h" /* MP_DECIMAL, MP_UUID */ +#include "lib/core/tt_static.h" +#include "lib/uuid/tt_uuid.h" /* tt_uuid_to_string(), UUID_STR_LEN */ #define DEFAULT_ENCODE_KEEP_BUFFER 1 @@ -421,15 +424,21 @@ static void json_append_data(lua_State *l, struct luaL_serializer *cfg, json_append_array(l, cfg, current_depth + 1, json, field.size); return; case MP_EXT: - switch (field.ext_type) { - case MP_DECIMAL: - { - const char *str = decimal_to_string(field.decval); - return json_append_string(cfg, json, str, strlen(str)); - } - default: - assert(false); - } + switch (field.ext_type) { + case MP_DECIMAL: + { + const char *str = decimal_to_string(field.decval); + return json_append_string(cfg, json, str, strlen(str)); + } + case MP_UUID: + { + char *str = tt_static_buf(); + tt_uuid_to_string(field.uuidval, str); + return json_append_string(cfg, json, str, UUID_STR_LEN); + } + default: + assert(false); + } } } diff --git a/third_party/lua-yaml/lyaml.cc b/third_party/lua-yaml/lyaml.cc index af4f2f5d5..411c56f71 100644 --- a/third_party/lua-yaml/lyaml.cc +++ b/third_party/lua-yaml/lyaml.cc @@ -50,6 +50,9 @@ extern "C" { } /* extern "C" */ #include "lua/utils.h" #include "lib/core/decimal.h" +#include "lib/core/tt_static.h" +#include "lib/core/mp_extension_types.h" /* MP_DECIMAL, MP_UUID */ +#include "lib/uuid/tt_uuid.h" /* tt_uuid_to_string(), UUID_STR_LEN */ #define LUAYAML_TAG_PREFIX "tag:yaml.org,2002:" @@ -697,10 +700,18 @@ static int dump_node(struct lua_yaml_dumper *dumper) switch (field.ext_type) { case MP_DECIMAL: str = decimal_to_string(field.decval); - len = strlen(str); - break; + len = strlen(str); + break; + case MP_UUID: + { + char *buf = tt_static_buf(); + tt_uuid_to_string(field.uuidval, buf); + str = buf; + len = UUID_STR_LEN; + break; + } default: - assert(0); /* checked by luaL_checkfield() */ + assert(0); /* checked by luaL_checkfield() */ } break; } -- 2.21.1 (Apple Git-122.3)