From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 47F3A6EC5F; Mon, 2 Aug 2021 03:44:12 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 47F3A6EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1627865052; bh=5LSCc8CDAxrC841GR5yO2pJWzmWLc1x/ADCQGVrp3v0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=nON2rJQ4JnFPpUAtziWrC6HhTcl/Hq9qQemRwTvqfiiNoPjL1PXdah7JyQXPXiQbY g1tQyJsfrMbQwBLIyt0pefdNhXJBSQFEO4E5hbrvb9Av5xxJSo0ZACA0YKuGqczpoR +5NFfT8Y+aFxXwMHXJm+UwJDigwJeBDpcAS8B+qE= Received: from smtp58.i.mail.ru (smtp58.i.mail.ru [217.69.128.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 152AA6EC61 for ; Mon, 2 Aug 2021 03:41:34 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 152AA6EC61 Received: by smtp58.i.mail.ru with esmtpa (envelope-from ) id 1mAM1M-0000PY-JR; Mon, 02 Aug 2021 03:41:33 +0300 To: v.shpilevoy@tarantool.org, olegrok@tarantool.org Date: Mon, 2 Aug 2021 03:41:01 +0300 Message-Id: <652dc466ebae43d3ac541785b57864d6519a91c7.1627864075.git.tsafin@tarantool.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD941C43E597735A9C354866C15C72ED952BE56FFA0EFAF5B8C182A05F53808504007066A2F09CEE8284EABFD219F052104ADF34A627B816B25AFD7BFA8E18D187E X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE746D93DAA4671895CEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AC18FED211962C318638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D88984FDADB7F8D2F9B6102A996DF06A67117882F4460429724CE54428C33FAD305F5C1EE8F4F765FCAA867293B0326636D2E47CDBA5A96583BD4B6F7A4D31EC0BC014FD901B82EE079FA2833FD35BB23D27C277FBC8AE2E8BF1175FABE1C0F9B6A471835C12D1D977C4224003CC836476EB9C4185024447017B076A6E789B0E975F5C1EE8F4F765FC7E8BD45E52F5999B3AA81AA40904B5D9CF19DD082D7633A078D18283394535A93AA81AA40904B5D98AA50765F790063715C16F4D43E65A23D81D268191BDAD3D698AB9A7B718F8C4D1B931868CE1C5781A620F70A64A45A98AA50765F79006372E808ACE2090B5E1725E5C173C3A84C3C5EA940A35A165FF2DBA43225CD8A89F83C798A30B85E16BC6EABA9B74D0DA47B5C8C57E37DE458BEDA766A37F9254B7 X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458F0AFF96BAACF4158235E5A14AD4A4A4625E192CAD1D9E79DB53CE84373687089BB272B0713E04AC2 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C85811CA530B504917BB2534926AE3F2EACADE5AA87702C309C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF03D88DA7F5FEDC25699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D340FE9920E7E3E5C6225F6831BBDBF8E998131929DDAD3311969BE25AAD60A3B56D9FCD7B2F01113621D7E09C32AA3244C8C888A4420E52A9DA8F7CF3D9BE649BAA95CA90A1D8AC5658D5DD81C2BAB7D1D X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojMCfuYI4Prec+f4P56M+KSg== X-Mailru-Sender: B5B6A6EBBD94DAD8315E1362B7B20B88AF73D57F25BEAD9680DADE4B61A702D0405E3F406E55B9F65C2808D6142752370A8ED71B308007E3DC85537438B7E1A423D748DE48713E689437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Timur Safin via Tarantool-patches Reply-To: Timur Safin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Serialize datetime_t as newly introduced MP_EXT type. It saves 1 required integer field and upto 2 optional unsigned fields in very compact fashion. - secs is required field; - but nsec, offset are both optional; * json, yaml serialization formats, lua output mode supported; * exported symbols for datetime messagepack size calculations so they are available for usage on Lua side. Part of #5941 Part of #5946 --- src/box/field_def.c | 34 +++--- src/box/field_def.h | 1 + src/box/lua/serialize_lua.c | 7 +- src/box/msgpack.c | 7 +- src/box/tuple_compare.cc | 20 ++++ src/exports.h | 2 + src/lib/core/CMakeLists.txt | 3 +- src/lib/core/datetime.c | 166 ++++++++++++++++++++++++++++++ src/lib/core/mp_extension_types.h | 1 + src/lib/mpstream/mpstream.c | 11 ++ src/lib/mpstream/mpstream.h | 4 + src/lua/msgpack.c | 12 +++ src/lua/msgpackffi.lua | 18 ++++ src/lua/serializer.c | 4 + src/lua/serializer.h | 2 + src/lua/utils.c | 1 - test/unit/datetime.c | 1 - third_party/lua-cjson/lua_cjson.c | 8 ++ third_party/lua-yaml/lyaml.cc | 6 +- 19 files changed, 287 insertions(+), 21 deletions(-) diff --git a/src/box/field_def.c b/src/box/field_def.c index 51acb8025..6964e3e9f 100644 --- a/src/box/field_def.c +++ b/src/box/field_def.c @@ -67,11 +67,12 @@ const uint32_t field_mp_type[] = { /* [FIELD_TYPE_VARBINARY] = */ 1U << MP_BIN, /* [FIELD_TYPE_SCALAR] = */ (1U << MP_UINT) | (1U << MP_INT) | (1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_STR) | - (1U << MP_BIN) | (1U << MP_BOOL), + (1U << MP_BIN) | (1U << MP_BOOL) | (1U << MP_DATETIME), /* [FIELD_TYPE_DECIMAL] = */ 0, /* only MP_DECIMAL is supported */ /* [FIELD_TYPE_UUID] = */ 0, /* only MP_UUID is supported */ /* [FIELD_TYPE_ARRAY] = */ 1U << MP_ARRAY, /* [FIELD_TYPE_MAP] = */ (1U << MP_MAP), + /* [FIELD_TYPE_DATETIME] = */ 0, /* only MP_DATETIME is supported */ }; const uint32_t field_ext_type[] = { @@ -88,6 +89,7 @@ const uint32_t field_ext_type[] = { /* [FIELD_TYPE_UUID] = */ 1U << MP_UUID, /* [FIELD_TYPE_ARRAY] = */ 0, /* [FIELD_TYPE_MAP] = */ 0, + /* [FIELD_TYPE_DATETIME] = */ 1U << MP_DATETIME, }; const char *field_type_strs[] = { @@ -104,6 +106,7 @@ const char *field_type_strs[] = { /* [FIELD_TYPE_UUID] = */ "uuid", /* [FIELD_TYPE_ARRAY] = */ "array", /* [FIELD_TYPE_MAP] = */ "map", + /* [FIELD_TYPE_DATETIME] = */ "datetime", }; const char *on_conflict_action_strs[] = { @@ -128,20 +131,21 @@ field_type_by_name_wrapper(const char *str, uint32_t len) * values can be stored in the j type. */ static const bool field_type_compatibility[] = { - /* ANY UNSIGNED STRING NUMBER DOUBLE INTEGER BOOLEAN VARBINARY SCALAR DECIMAL UUID ARRAY MAP */ -/* ANY */ true, false, false, false, false, false, false, false, false, false, false, false, false, -/* UNSIGNED */ true, true, false, true, false, true, false, false, true, false, false, false, false, -/* STRING */ true, false, true, false, false, false, false, false, true, false, false, false, false, -/* NUMBER */ true, false, false, true, false, false, false, false, true, false, false, false, false, -/* DOUBLE */ true, false, false, true, true, false, false, false, true, false, false, false, false, -/* INTEGER */ true, false, false, true, false, true, false, false, true, false, false, false, false, -/* BOOLEAN */ true, false, false, false, false, false, true, false, true, false, false, false, false, -/* VARBINARY*/ true, false, false, false, false, false, false, true, true, false, false, false, false, -/* SCALAR */ true, false, false, false, false, false, false, false, true, false, false, false, false, -/* DECIMAL */ true, false, false, true, false, false, false, false, true, true, false, false, false, -/* UUID */ true, false, false, false, false, false, false, false, false, false, true, false, false, -/* ARRAY */ true, false, false, false, false, false, false, false, false, false, false, true, false, -/* MAP */ true, false, false, false, false, false, false, false, false, false, false, false, true, + /* ANY UNSIGNED STRING NUMBER DOUBLE INTEGER BOOLEAN VARBINARY SCALAR DECIMAL UUID ARRAY MAP DATETIME */ +/* ANY */ true, false, false, false, false, false, false, false, false, false, false, false, false, false, +/* UNSIGNED */ true, true, false, true, false, true, false, false, true, false, false, false, false, false, +/* STRING */ true, false, true, false, false, false, false, false, true, false, false, false, false, false, +/* NUMBER */ true, false, false, true, false, false, false, false, true, false, false, false, false, false, +/* DOUBLE */ true, false, false, true, true, false, false, false, true, false, false, false, false, false, +/* INTEGER */ true, false, false, true, false, true, false, false, true, false, false, false, false, false, +/* BOOLEAN */ true, false, false, false, false, false, true, false, true, false, false, false, false, false, +/* VARBINARY*/ true, false, false, false, false, false, false, true, true, false, false, false, false, false, +/* SCALAR */ true, false, false, false, false, false, false, false, true, false, false, false, false, false, +/* DECIMAL */ true, false, false, true, false, false, false, false, true, true, false, false, false, false, +/* UUID */ true, false, false, false, false, false, false, false, false, false, true, false, false, false, +/* ARRAY */ true, false, false, false, false, false, false, false, false, false, false, true, false, false, +/* MAP */ true, false, false, false, false, false, false, false, false, false, false, false, true, false, +/* DATETIME */ true, false, false, false, false, false, false, false, false, false, false, false, false, true, }; bool diff --git a/src/box/field_def.h b/src/box/field_def.h index c5cfe5e86..120b2a93d 100644 --- a/src/box/field_def.h +++ b/src/box/field_def.h @@ -63,6 +63,7 @@ enum field_type { FIELD_TYPE_UUID, FIELD_TYPE_ARRAY, FIELD_TYPE_MAP, + FIELD_TYPE_DATETIME, field_type_MAX }; diff --git a/src/box/lua/serialize_lua.c b/src/box/lua/serialize_lua.c index 7144305cf..5cfa6f5e0 100644 --- a/src/box/lua/serialize_lua.c +++ b/src/box/lua/serialize_lua.c @@ -768,7 +768,7 @@ static int dump_node(struct lua_dumper *d, struct node *nd, int indent) { struct luaL_field *field = &nd->field; - char buf[FPCONV_G_FMT_BUFSIZE]; + char buf[FPCONV_G_FMT_BUFSIZE + 8]; int ltype = lua_type(d->L, -1); const char *str = NULL; size_t len = 0; @@ -861,6 +861,11 @@ dump_node(struct lua_dumper *d, struct node *nd, int indent) str = tt_uuid_str(field->uuidval); len = UUID_STR_LEN; break; + case MP_DATETIME: + nd->mask |= NODE_QUOTE; + str = buf; + len = datetime_to_string(field->dateval, buf, sizeof buf); + break; default: d->err = EINVAL; snprintf(d->err_msg, sizeof(d->err_msg), diff --git a/src/box/msgpack.c b/src/box/msgpack.c index 1723dea4c..e53af548c 100644 --- a/src/box/msgpack.c +++ b/src/box/msgpack.c @@ -1,5 +1,5 @@ /* - * Copyright 2020, Tarantool AUTHORS, please see AUTHORS file. + * Copyright 2020-2021, Tarantool AUTHORS, please see AUTHORS file. * * Redistribution and use in source and binary forms, with or * without modification, are permitted provided that the following @@ -35,6 +35,7 @@ #include "mp_decimal.h" #include "uuid/mp_uuid.h" #include "mp_error.h" +#include "datetime.h" static int msgpack_fprint_ext(FILE *file, const char **data, int depth) @@ -47,6 +48,8 @@ msgpack_fprint_ext(FILE *file, const char **data, int depth) return mp_fprint_decimal(file, data, len); case MP_UUID: return mp_fprint_uuid(file, data, len); + case MP_DATETIME: + return mp_fprint_datetime(file, data, len); case MP_ERROR: return mp_fprint_error(file, data, depth); default: @@ -65,6 +68,8 @@ msgpack_snprint_ext(char *buf, int size, const char **data, int depth) return mp_snprint_decimal(buf, size, data, len); case MP_UUID: return mp_snprint_uuid(buf, size, data, len); + case MP_DATETIME: + return mp_snprint_datetime(buf, size, data, len); case MP_ERROR: return mp_snprint_error(buf, size, data, depth); default: diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc index 98938fb39..48c833643 100644 --- a/src/box/tuple_compare.cc +++ b/src/box/tuple_compare.cc @@ -36,6 +36,7 @@ #include "lib/core/decimal.h" #include "lib/core/mp_decimal.h" #include "uuid/mp_uuid.h" +#include "core/datetime.h" #include "lib/core/mp_extension_types.h" /* {{{ tuple_compare */ @@ -76,6 +77,7 @@ enum mp_class { MP_CLASS_STR, MP_CLASS_BIN, MP_CLASS_UUID, + MP_CLASS_DATETIME, MP_CLASS_ARRAY, MP_CLASS_MAP, mp_class_max, @@ -99,6 +101,8 @@ static enum mp_class mp_ext_classes[] = { /* .MP_UNKNOWN_EXTENSION = */ mp_class_max, /* unsupported */ /* .MP_DECIMAL = */ MP_CLASS_NUMBER, /* .MP_UUID = */ MP_CLASS_UUID, + /* .MP_ERROR = */ mp_class_max, + /* .MP_DATETIME = */ MP_CLASS_DATETIME, }; static enum mp_class @@ -390,6 +394,19 @@ mp_compare_uuid(const char *field_a, const char *field_b) return memcmp(field_a + 2, field_b + 2, UUID_PACKED_LEN); } +static int +mp_compare_datetime(const char *lhs, const char *rhs) +{ + datetime_t lhs_dt, rhs_dt; + datetime_t *ret; + ret = mp_decode_datetime(&lhs, &lhs_dt); + assert(ret != NULL); + ret = mp_decode_datetime(&rhs, &rhs_dt); + assert(ret != NULL); + (void)ret; + return datetime_compare(&lhs_dt, &rhs_dt); +} + typedef int (*mp_compare_f)(const char *, const char *); static mp_compare_f mp_class_comparators[] = { /* .MP_CLASS_NIL = */ NULL, @@ -398,6 +415,7 @@ static mp_compare_f mp_class_comparators[] = { /* .MP_CLASS_STR = */ mp_compare_str, /* .MP_CLASS_BIN = */ mp_compare_bin, /* .MP_CLASS_UUID = */ mp_compare_uuid, + /* .MP_CLASS_DATETIME=*/ mp_compare_datetime, /* .MP_CLASS_ARRAY = */ NULL, /* .MP_CLASS_MAP = */ NULL, }; @@ -478,6 +496,8 @@ tuple_compare_field(const char *field_a, const char *field_b, return mp_compare_decimal(field_a, field_b); case FIELD_TYPE_UUID: return mp_compare_uuid(field_a, field_b); + case FIELD_TYPE_DATETIME: + return mp_compare_datetime(field_a, field_b); default: unreachable(); return 0; diff --git a/src/exports.h b/src/exports.h index ce8f74065..7ac2cd012 100644 --- a/src/exports.h +++ b/src/exports.h @@ -459,6 +459,7 @@ EXPORT(mp_encode_decimal) EXPORT(mp_encode_double) EXPORT(mp_encode_float) EXPORT(mp_encode_uuid) +EXPORT(mp_encode_datetime) EXPORT(mp_ext_hint) EXPORT(mp_format) EXPORT(mp_fprint) @@ -468,6 +469,7 @@ EXPORT(mp_fprint_recursion) EXPORT(mp_parser_hint) EXPORT(mp_sizeof_decimal) EXPORT(mp_sizeof_uuid) +EXPORT(mp_sizeof_datetime) EXPORT(mp_snprint) EXPORT(mp_snprint_ext) EXPORT(mp_snprint_ext_default) diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt index 8bc776b82..c3e68c36f 100644 --- a/src/lib/core/CMakeLists.txt +++ b/src/lib/core/CMakeLists.txt @@ -44,7 +44,8 @@ add_library(core STATIC ${core_sources}) target_link_libraries(core salad small uri decNumber bit ${LIBEV_LIBRARIES} ${LIBEIO_LIBRARIES} ${LIBCORO_LIBRARIES} - ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES}) + ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES} + ${LIBCDT_LIBRARIES}) if (ENABLE_BACKTRACE AND NOT TARGET_OS_DARWIN) target_link_libraries(core gcc_s ${UNWIND_LIBRARIES}) diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c index 65f813a70..96e554bd2 100755 --- a/src/lib/core/datetime.c +++ b/src/lib/core/datetime.c @@ -33,6 +33,137 @@ #include "trivia/util.h" #include "datetime.h" +#include "msgpuck.h" +#include "mp_extension_types.h" + +/* + Datetime MessagePack serialization schema is MP_EXT (0xC7 for 1 byte length) + extension, which creates container of 1 to 3 integers. + + +----+---+-----------+====~~~~~~~====+-----~~~~~~~~-------+....~~~~~~~....+ + |0xC7| 4 |len (uint8)| seconds (int) | nanoseconds (uint) | offset (uint) | + +----+---+-----------+====~~~~~~~====+-----~~~~~~~~-------+....~~~~~~~....+ + + MessagePack extension MP_EXT (0xC7), after 1-byte length, contains: + + - signed integer seconds part (required). Depending on the value of + seconds it may be from 1 to 8 bytes positive or negative integer number; + + - [optional] fraction time in nanoseconds as unsigned integer. + If this value is 0 then it's not saved (unless there is offset field, + as below); + + - [optional] timzeone offset in minutes as unsigned integer. + If this field is 0 then it's not saved. + */ + +static inline uint32_t +mp_sizeof_Xint(int64_t n) +{ + return n < 0 ? mp_sizeof_int(n) : mp_sizeof_uint(n); +} + +static inline char * +mp_encode_Xint(char *data, int64_t v) +{ + return v < 0 ? mp_encode_int(data, v) : mp_encode_uint(data, v); +} + +static inline int64_t +mp_decode_Xint(const char **data) +{ + switch (mp_typeof(**data)) { + case MP_UINT: + return (int64_t)mp_decode_uint(data); + case MP_INT: + return mp_decode_int(data); + default: + mp_unreachable(); + } + return 0; +} + +static inline uint32_t +mp_sizeof_datetime_raw(const struct datetime_t *date) +{ + uint32_t sz = mp_sizeof_Xint(date->secs); + + // even if nanosecs == 0 we need to output anything + // if we have non-null tz offset + if (date->nsec != 0 || date->offset != 0) + sz += mp_sizeof_Xint(date->nsec); + if (date->offset) + sz += mp_sizeof_Xint(date->offset); + return sz; +} + +uint32_t +mp_sizeof_datetime(const struct datetime_t *date) +{ + return mp_sizeof_ext(mp_sizeof_datetime_raw(date)); +} + +struct datetime_t * +datetime_unpack(const char **data, uint32_t len, struct datetime_t *date) +{ + const char * svp = *data; + + memset(date, 0, sizeof(*date)); + + date->secs = mp_decode_Xint(data); + + len -= *data - svp; + if (len <= 0) + return date; + + svp = *data; + date->nsec = mp_decode_Xint(data); + len -= *data - svp; + + if (len <= 0) + return date; + + date->offset = mp_decode_Xint(data); + + return date; +} + +struct datetime_t * +mp_decode_datetime(const char **data, struct datetime_t *date) +{ + if (mp_typeof(**data) != MP_EXT) + return NULL; + + int8_t type; + uint32_t len = mp_decode_extl(data, &type); + + if (type != MP_DATETIME || len == 0) { + return NULL; + } + return datetime_unpack(data, len, date); +} + +char * +datetime_pack(char *data, const struct datetime_t *date) +{ + data = mp_encode_Xint(data, date->secs); + if (date->nsec != 0 || date->offset != 0) + data = mp_encode_Xint(data, date->nsec); + if (date->offset) + data = mp_encode_Xint(data, date->offset); + + return data; +} + +char * +mp_encode_datetime(char *data, const struct datetime_t *date) +{ + uint32_t len = mp_sizeof_datetime_raw(date); + + data = mp_encode_extl(data, MP_DATETIME, len); + + return datetime_pack(data, date); +} int datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len) @@ -83,3 +214,38 @@ datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len) } return (buf - src); } + +int +mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len) +{ + struct datetime_t date = {0}; + + if (datetime_unpack(data, len, &date) == NULL) + return -1; + + return datetime_to_string(&date, buf, size); +} + +int +mp_fprint_datetime(FILE *file, const char **data, uint32_t len) +{ + struct datetime_t date; + + if (datetime_unpack(data, len, &date) == NULL) + return -1; + + char buf[48]; + datetime_to_string(&date, buf, sizeof buf); + + return fprintf(file, "%s", buf); +} + +int datetime_compare(const struct datetime_t *lhs, const struct datetime_t *rhs) +{ + int result = COMPARE_RESULT(lhs->secs, rhs->secs); + if (result != 0) + return result; + + return COMPARE_RESULT(lhs->nsec, rhs->nsec); +} + diff --git a/src/lib/core/mp_extension_types.h b/src/lib/core/mp_extension_types.h index e3ff9f5d0..3b7eaee7c 100644 --- a/src/lib/core/mp_extension_types.h +++ b/src/lib/core/mp_extension_types.h @@ -44,6 +44,7 @@ enum mp_extension_type { MP_DECIMAL = 1, MP_UUID = 2, MP_ERROR = 3, + MP_DATETIME = 4, mp_extension_type_MAX, }; diff --git a/src/lib/mpstream/mpstream.c b/src/lib/mpstream/mpstream.c index 70ca29889..1077e3b19 100644 --- a/src/lib/mpstream/mpstream.c +++ b/src/lib/mpstream/mpstream.c @@ -35,6 +35,7 @@ #include "msgpuck.h" #include "mp_decimal.h" #include "uuid/mp_uuid.h" +#include "core/datetime.h" void mpstream_reserve_slow(struct mpstream *stream, size_t size) @@ -208,6 +209,16 @@ mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid) mpstream_advance(stream, pos - data); } +void +mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *val) +{ + char *data = mpstream_reserve(stream, mp_sizeof_datetime(val)); + if (data == NULL) + return; + char *pos = mp_encode_datetime(data, val); + mpstream_advance(stream, pos - data); +} + void mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n) { diff --git a/src/lib/mpstream/mpstream.h b/src/lib/mpstream/mpstream.h index a60add143..540e9a666 100644 --- a/src/lib/mpstream/mpstream.h +++ b/src/lib/mpstream/mpstream.h @@ -39,6 +39,7 @@ extern "C" { #endif /* defined(__cplusplus) */ struct tt_uuid; +struct datetime_t; /** * Ask the allocator to reserve at least size bytes. It can reserve @@ -145,6 +146,9 @@ mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val); void mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid); +void +mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *dt); + /** Copies n bytes from memory area src to stream. */ void mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n); diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c index b6ecf2b1e..0a4ba8129 100644 --- a/src/lua/msgpack.c +++ b/src/lua/msgpack.c @@ -46,6 +46,7 @@ #include "lib/core/decimal.h" /* decimal_unpack() */ #include "lib/uuid/mp_uuid.h" /* mp_decode_uuid() */ #include "lib/core/mp_extension_types.h" +#include "datetime.h" #include "cord_buf.h" #include @@ -200,6 +201,9 @@ restart: /* used by MP_EXT of unidentified subtype */ break; case MP_ERROR: return luamp_encode_extension(L, top, stream); + case MP_DATETIME: + mpstream_encode_datetime(stream, field->dateval); + break; default: /* Run trigger if type can't be encoded */ type = luamp_encode_extension(L, top, stream); @@ -333,6 +337,14 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg, goto ext_decode_err; return; } + case MP_DATETIME: + { + struct datetime_t * date = luaL_pushdatetime(L); + date = datetime_unpack(data, len, date); + if (date == NULL) + goto ext_decode_err; + return; + } default: /* reset data to the extension header */ *data = svp; diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua index 1d54f11b8..c47d77acb 100644 --- a/src/lua/msgpackffi.lua +++ b/src/lua/msgpackffi.lua @@ -26,6 +26,10 @@ char * mp_encode_uuid(char *data, const struct tt_uuid *uuid); uint32_t mp_sizeof_uuid(); +uint32_t +mp_sizeof_datetime(const struct t_datetime_tz *date); +char * +mp_encode_datetime(char *data, const struct t_datetime_tz *date); float mp_decode_float(const char **data); double @@ -36,6 +40,8 @@ decimal_t * decimal_unpack(const char **data, uint32_t len, decimal_t *dec); struct tt_uuid * uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid); +struct datetime_t * +datetime_unpack(const char **data, uint32_t len, struct datetime_t *date); ]]) local strict_alignment = (jit.arch == 'arm') @@ -142,6 +148,11 @@ local function encode_uuid(buf, uuid) builtin.mp_encode_uuid(p, uuid) end +local function encode_datetime(buf, date) + local p = buf:alloc(builtin.mp_sizeof_datetime(date)) + builtin.mp_encode_datetime(p, date) +end + local function encode_int(buf, num) if num >= 0 then if num <= 0x7f then @@ -320,6 +331,7 @@ on_encode(ffi.typeof('float'), encode_float) on_encode(ffi.typeof('double'), encode_double) on_encode(ffi.typeof('decimal_t'), encode_decimal) on_encode(ffi.typeof('struct tt_uuid'), encode_uuid) +on_encode(ffi.typeof('struct datetime_t'), encode_datetime) -------------------------------------------------------------------------------- -- Decoder @@ -513,6 +525,12 @@ local ext_decoder = { builtin.uuid_unpack(data, len, uuid) return uuid end, + -- MP_DATETIME + [4] = function(data, len) + local dt = ffi.new("struct datetime_t") + builtin.datetime_unpack(data, len, dt) + return dt + end, } local function decode_ext(data) diff --git a/src/lua/serializer.c b/src/lua/serializer.c index 8db6746a3..c27e62c62 100644 --- a/src/lua/serializer.c +++ b/src/lua/serializer.c @@ -41,6 +41,7 @@ #include "lib/core/mp_extension_types.h" #include "lua/error.h" +#include "datetime.h" #include "trivia/util.h" #include "diag.h" #include "serializer_opts.h" @@ -544,6 +545,9 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, opts != NULL && opts->error_marshaling_enabled) { field->ext_type = MP_ERROR; + } else if (cd->ctypeid == CTID_DATETIME) { + field->ext_type = MP_DATETIME; + field->dateval = (struct datetime_t*) cdata; } else { field->ext_type = MP_UNKNOWN_EXTENSION; } diff --git a/src/lua/serializer.h b/src/lua/serializer.h index 0a0501a74..52e51d279 100644 --- a/src/lua/serializer.h +++ b/src/lua/serializer.h @@ -52,6 +52,7 @@ extern "C" { #include #include "trigger.h" +#include "lib/core/datetime.h" #include "lib/core/decimal.h" /* decimal_t */ #include "lib/core/mp_extension_types.h" #include "lua/error.h" @@ -223,6 +224,7 @@ struct luaL_field { uint32_t size; decimal_t *decval; struct tt_uuid *uuidval; + struct datetime_t *dateval; }; enum mp_type type; /* subtypes of MP_EXT */ diff --git a/src/lua/utils.c b/src/lua/utils.c index 611044b6f..685cab47b 100644 --- a/src/lua/utils.c +++ b/src/lua/utils.c @@ -241,7 +241,6 @@ luaL_setcdatagc(struct lua_State *L, int idx) lua_pop(L, 1); } - /** * A helper to register a single type metatable. */ diff --git a/test/unit/datetime.c b/test/unit/datetime.c index b6f568c03..226b6fadb 100644 --- a/test/unit/datetime.c +++ b/test/unit/datetime.c @@ -139,7 +139,6 @@ exit: // avoid introducing external datetime.h dependency // - just copy paste it for today #define SECS_PER_DAY 86400 -#define NANOS_PER_SEC 1000000000 #define DT_EPOCH_1970_OFFSET 719163 diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c index 5123b9a74..d5ddc4ea0 100644 --- a/third_party/lua-cjson/lua_cjson.c +++ b/third_party/lua-cjson/lua_cjson.c @@ -52,6 +52,7 @@ #include "mp_extension_types.h" /* MP_DECIMAL, MP_UUID */ #include "tt_static.h" #include "uuid/tt_uuid.h" /* tt_uuid_to_string(), UUID_STR_LEN */ +#include "core/datetime.h" #include "cord_buf.h" typedef enum { @@ -426,6 +427,13 @@ static void json_append_data(lua_State *l, struct luaL_serializer *cfg, case MP_UUID: return json_append_string(cfg, json, tt_uuid_str(field.uuidval), UUID_STR_LEN); + + case MP_DATETIME: + { + char buf[128]; + size_t sz = datetime_to_string(field.dateval, buf, sizeof buf); + return json_append_string(cfg, json, buf, sz); + } default: assert(false); } diff --git a/third_party/lua-yaml/lyaml.cc b/third_party/lua-yaml/lyaml.cc index 5469e9f4f..39127fccf 100644 --- a/third_party/lua-yaml/lyaml.cc +++ b/third_party/lua-yaml/lyaml.cc @@ -617,7 +617,7 @@ static int dump_node(struct lua_yaml_dumper *dumper) yaml_event_t ev; yaml_scalar_style_t style = YAML_PLAIN_SCALAR_STYLE; int is_binary = 0; - char buf[FPCONV_G_FMT_BUFSIZE]; + char buf[FPCONV_G_FMT_BUFSIZE + 8]; struct luaL_field field; bool unused; (void) unused; @@ -707,6 +707,10 @@ static int dump_node(struct lua_yaml_dumper *dumper) str = tt_uuid_str(field.uuidval); len = UUID_STR_LEN; break; + case MP_DATETIME: + len = datetime_to_string(field.dateval, buf, sizeof buf); + str = buf; + break; default: assert(0); /* checked by luaL_checkfield() */ } -- 2.29.2