Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v3 0/9] Initial datetime support
@ 2021-08-02  0:40 Timur Safin via Tarantool-patches
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build Timur Safin via Tarantool-patches
                   ` (11 more replies)
  0 siblings, 12 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:40 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* Version #3 changes:

  - renamed `struct datetime_t` to `struct datetime`, and `struct 
    datetime_interval_t` to `struct datetime_interval`;
  - significantly reworked arguments checks in module api entries;
  - fixed datetime comparisons;
  - changed hints calculation to take into account fractional part;
  - provided more comments here and there;

NB! There are MacOSX problems due to GLIBC specific code used (as Vlad
    has already pointed out) - so additional patch, making it more 
    cross-compatible coming here soon...

* Version #2 changes:

  - fixed problem with overloaded '-' and '+' operations for datetime 
    arguments;
  - fixed messagepack serialization problems;
  - heavily documented MessagePack serialization schema in the code;
  - introduced working implementation of datetime hints for storage engines;
  - made interval related names be more consistent, renamed durations and period 
    to intervals, i.e. t_datetime_duration to datetime_interval_t, 
    duration_* to interval_*, period to interval;
  - properly implemented all reasonable cases of datetime+interval 
    arithmetic;
  - moved all initialization code to utils.c;
  - renamed core/mp_datetime.c to core/datetime.c because it makes more
    sense now;

* Version #1 - initial RFC series

In brief
--------
This patchset implements datetime lua support in box, with serialization 
to messagepack, yaml, json and lua mode. Also it contains storage 
engines' indices implementation for datetime type introduced.

* Current implementation is heavily influenced by Sci-Lua lua-time module
  https://github.com/stepelu/lua-time 
  e.g. you could find very similar approach for handling of operations
  with year or month long intervals (which should be handled differently than
  usual intervals of seconds, or days).

* But internally we actually use Christian Hanson' c-dt module
  https://github.com/chansen/c-dt 
  (though it has been modified slightly for cleaner integration
  into cmake build process)


Datetime Module API
-------------------

We used to have here draft documentation of datetime module api, but 
for a convenience it has been extracted to the discussion topic there - 
https://github.com/tarantool/tarantool/discussions/6244#discussioncomment-1043988

Messagepack serialization schema
--------------------------------

In short it looks like:
- now we introduce new MP_EXT extension type #4;
- we may save 1 required and 2 optional fields for datetime field;

In all gory details it's explained in MessagePack serialization schema depicted here:
https://github.com/tarantool/tarantool/discussions/6244#discussioncomment-1043990


https://github.com/tarantool/tarantool/issues/5941
https://github.com/tarantool/tarantool/issues/5946

https://github.com/tarantool/tarantool/tree/tsafin/gh-5941-datetime-v3

Timur Safin (9):
  build: add Christian Hansen c-dt to the build
  lua: built-in module datetime
  lua, datetime: datetime tests
  lua, datetime: display datetime
  box, datetime: messagepack support for datetime
  box, datetime: datetime comparison for indices
  lua, datetime: time intervals support
  datetime: changelog for datetime module
  lua, box, datetime: rename struct datetime_t

 .gitmodules                                   |    3 +
 CMakeLists.txt                                |    8 +
 .../gh-5941-datetime-type-support.md          |    4 +
 cmake/BuildCDT.cmake                          |    6 +
 src/CMakeLists.txt                            |    3 +
 src/box/field_def.c                           |   52 +-
 src/box/field_def.h                           |    4 +
 src/box/lua/serialize_lua.c                   |    7 +-
 src/box/memtx_space.c                         |    3 +-
 src/box/msgpack.c                             |    7 +-
 src/box/tuple_compare.cc                      |   76 ++
 src/box/vinyl.c                               |    3 +-
 src/exports.h                                 |   29 +
 src/lib/core/CMakeLists.txt                   |    4 +-
 src/lib/core/datetime.c                       |  250 ++++
 src/lib/core/datetime.h                       |  115 ++
 src/lib/core/mp_extension_types.h             |    1 +
 src/lib/mpstream/mpstream.c                   |   10 +
 src/lib/mpstream/mpstream.h                   |    3 +
 src/lua/datetime.lua                          | 1058 +++++++++++++++++
 src/lua/init.c                                |    4 +-
 src/lua/msgpack.c                             |   12 +
 src/lua/msgpackffi.lua                        |   18 +
 src/lua/serializer.c                          |    4 +
 src/lua/serializer.h                          |    2 +
 src/lua/utils.c                               |   28 +-
 src/lua/utils.h                               |   12 +
 test/app-tap/datetime.test.lua                |  367 ++++++
 test/engine/datetime.result                   |   77 ++
 test/engine/datetime.test.lua                 |   35 +
 test/unit/CMakeLists.txt                      |    2 +
 test/unit/datetime.c                          |  221 ++++
 test/unit/datetime.result                     |  358 ++++++
 third_party/c-dt                              |    1 +
 third_party/lua-cjson/lua_cjson.c             |    8 +
 third_party/lua-yaml/lyaml.cc                 |    6 +-
 36 files changed, 2776 insertions(+), 25 deletions(-)
 create mode 100644 changelogs/unreleased/gh-5941-datetime-type-support.md
 create mode 100644 cmake/BuildCDT.cmake
 create mode 100755 src/lib/core/datetime.c
 create mode 100644 src/lib/core/datetime.h
 create mode 100644 src/lua/datetime.lua
 create mode 100755 test/app-tap/datetime.test.lua
 create mode 100644 test/engine/datetime.result
 create mode 100644 test/engine/datetime.test.lua
 create mode 100644 test/unit/datetime.c
 create mode 100644 test/unit/datetime.result
 create mode 160000 third_party/c-dt

-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
@ 2021-08-02  0:40 ` Timur Safin via Tarantool-patches
  2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime Timur Safin via Tarantool-patches
                   ` (10 subsequent siblings)
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:40 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* Integrated chansen/c-dt parser as 3rd party module to the
  Tarantool cmake build process.
* Points to tsafin/c-dt instead iof original chansen/c-dt to
  have easier build integration, because there is additional
  commit which integrated cmake support
---
 .gitmodules          | 3 +++
 CMakeLists.txt       | 8 ++++++++
 cmake/BuildCDT.cmake | 6 ++++++
 src/CMakeLists.txt   | 1 +
 third_party/c-dt     | 1 +
 5 files changed, 19 insertions(+)
 create mode 100644 cmake/BuildCDT.cmake
 create mode 160000 third_party/c-dt

diff --git a/.gitmodules b/.gitmodules
index f2f91ee72..aa3fbae4e 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -43,3 +43,6 @@
 [submodule "third_party/xxHash"]
 	path = third_party/xxHash
 	url = https://github.com/tarantool/xxHash
+[submodule "third_party/c-dt"]
+	path = third_party/c-dt
+	url = https://github.com/tarantool/c-dt.git
diff --git a/CMakeLists.txt b/CMakeLists.txt
index f9e1a7f79..81ee92933 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -571,6 +571,14 @@ endif()
 # zstd
 #
 
+#
+# Chritian Hanson c-dt
+#
+
+include(BuildCDT)
+libccdt_build()
+add_dependencies(build_bundled_libs cdt)
+
 #
 # Third-Party misc
 #
diff --git a/cmake/BuildCDT.cmake b/cmake/BuildCDT.cmake
new file mode 100644
index 000000000..a19abb35c
--- /dev/null
+++ b/cmake/BuildCDT.cmake
@@ -0,0 +1,6 @@
+macro(libccdt_build)
+    set(LIBCDT_INCLUDE_DIRS ${PROJECT_SOURCE_DIR}/third_party/c-dt/)
+    set(LIBCDT_LIBRARIES cdt)
+
+    add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/c-dt)
+endmacro()
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index c0e272bd9..ef6a295d5 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -202,6 +202,7 @@ set (common_libraries
     ${ICONV_LIBRARIES}
     ${OPENSSL_LIBRARIES}
     ${XXHASH_LIBRARIES}
+    ${LIBCDT_LIBRARIES}
 )
 
 if (TARGET_OS_LINUX OR TARGET_OS_DEBIAN_FREEBSD)
diff --git a/third_party/c-dt b/third_party/c-dt
new file mode 160000
index 000000000..8b61c4ea0
--- /dev/null
+++ b/third_party/c-dt
@@ -0,0 +1 @@
+Subproject commit 8b61c4ea006efefc3a068f8df4a156bf5c725c89
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build Timur Safin via Tarantool-patches
@ 2021-08-02  0:40 ` Timur Safin via Tarantool-patches
  2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
                     ` (2 more replies)
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 3/9] lua, datetime: datetime tests Timur Safin via Tarantool-patches
                   ` (9 subsequent siblings)
  11 siblings, 3 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:40 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* created a new Tarantool built-in module `datetime`;
* register cdef types for this module;
* export some `dt_*` functions from `c-dt` library;
* lua implementationis of `asctime` and `strftime`;
* datetime parsing unit tests, with and withput timezones;
* c test for reversible strftime roundtrip;

Part of #5941
---
 src/CMakeLists.txt        |   2 +
 src/exports.h             |  21 ++
 src/lib/core/datetime.h   | 112 ++++++
 src/lua/datetime.lua      | 756 ++++++++++++++++++++++++++++++++++++++
 src/lua/init.c            |   4 +-
 src/lua/utils.c           |  27 ++
 src/lua/utils.h           |  12 +
 test/unit/CMakeLists.txt  |   2 +
 test/unit/datetime.c      | 221 +++++++++++
 test/unit/datetime.result | 358 ++++++++++++++++++
 10 files changed, 1514 insertions(+), 1 deletion(-)
 create mode 100644 src/lib/core/datetime.h
 create mode 100644 src/lua/datetime.lua
 create mode 100644 test/unit/datetime.c
 create mode 100644 test/unit/datetime.result

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index ef6a295d5..e0499e57f 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -51,6 +51,8 @@ lua_source(lua_sources ../third_party/luafun/fun.lua)
 lua_source(lua_sources lua/httpc.lua)
 lua_source(lua_sources lua/iconv.lua)
 lua_source(lua_sources lua/swim.lua)
+lua_source(lua_sources lua/datetime.lua)
+
 # LuaJIT jit.* library
 lua_source(lua_sources ${LUAJIT_SOURCE_ROOT}/src/jit/bc.lua)
 lua_source(lua_sources ${LUAJIT_SOURCE_ROOT}/src/jit/bcsave.lua)
diff --git a/src/exports.h b/src/exports.h
index 5bb3e6a2b..2ec314369 100644
--- a/src/exports.h
+++ b/src/exports.h
@@ -218,6 +218,27 @@ EXPORT(curl_version)
 EXPORT(curl_version_info)
 #endif /* EXPORT_LIBCURL_SYMBOLS */
 EXPORT(decimal_unpack)
+EXPORT(dt_dow)
+EXPORT(dt_from_rdn)
+EXPORT(dt_from_struct_tm)
+EXPORT(dt_from_yd)
+EXPORT(dt_from_ymd)
+EXPORT(dt_from_yqd)
+EXPORT(dt_from_ywd)
+EXPORT(dt_parse_iso_date)
+EXPORT(dt_parse_iso_time)
+EXPORT(dt_parse_iso_time_basic)
+EXPORT(dt_parse_iso_time_extended)
+EXPORT(dt_parse_iso_zone)
+EXPORT(dt_parse_iso_zone_basic)
+EXPORT(dt_parse_iso_zone_extended)
+EXPORT(dt_parse_iso_zone_lenient)
+EXPORT(dt_rdn)
+EXPORT(dt_to_struct_tm)
+EXPORT(dt_to_yd)
+EXPORT(dt_to_ymd)
+EXPORT(dt_to_yqd)
+EXPORT(dt_to_ywd)
 EXPORT(error_ref)
 EXPORT(error_set_prev)
 EXPORT(error_unpack_unsafe)
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
new file mode 100644
index 000000000..84d6642ab
--- /dev/null
+++ b/src/lib/core/datetime.h
@@ -0,0 +1,112 @@
+#pragma once
+/*
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ *    copyright notice, this list of conditions and the
+ *    following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ *    copyright notice, this list of conditions and the following
+ *    disclaimer in the documentation and/or other materials
+ *    provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <stdint.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include "c-dt/dt.h"
+
+#if defined(__cplusplus)
+extern "C"
+{
+#endif /* defined(__cplusplus) */
+
+/**
+ * Full datetime structure representing moments
+ * since Unix Epoch (1970-01-01).
+ * Time is kept normalized to UTC, time-zone offset
+ * is informative only.
+ */
+struct datetime_t {
+	int64_t secs;	/**< seconds since epoch */
+	int32_t nsec;	/**< nanoseconds if any */
+	int32_t offset; /**< offset in minutes from UTC */
+};
+
+/**
+ * Date/time interval structure
+ */
+struct datetime_interval_t {
+	int64_t secs; /**< relative seconds delta */
+	int32_t nsec; /**< nanoseconds delta */
+};
+
+int
+datetime_compare(const struct datetime_t * lhs,
+		 const struct datetime_t * rhs);
+
+
+struct datetime_t *
+datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
+
+/**
+ * Pack datetime_t data to the MessagePack buffer.
+ */
+char *
+datetime_pack(char *data, const struct datetime_t *date);
+
+/**
+ * Calculate size of MessagePack buffer for datetime_t data.
+ */
+uint32_t
+mp_sizeof_datetime(const struct datetime_t *date);
+
+/**
+ * Decode data from MessagePack buffer to datetime_t structure.
+ */
+struct datetime_t *
+mp_decode_datetime(const char **data, struct datetime_t *date);
+
+/**
+ * Encode datetime_t structure to the MessagePack buffer.
+ */
+char *
+mp_encode_datetime(char *data, const struct datetime_t *date);
+
+/**
+ * Convert datetime to string using default format
+ * @param date source datetime value
+ * @param buf output character buffer
+ * @param len size ofoutput buffer
+ */
+int
+datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
+
+int
+mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
+
+int
+mp_fprint_datetime(FILE *file, const char **data, uint32_t len);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
new file mode 100644
index 000000000..870edc541
--- /dev/null
+++ b/src/lua/datetime.lua
@@ -0,0 +1,756 @@
+local ffi = require('ffi')
+
+ffi.cdef [[
+
+    /*
+    `c-dt` library functions handles properly both positive and negative `dt`
+    values, where `dt` is a number of dates since Rata Die date (0001-01-01).
+
+    For better compactness of our typical data in MessagePack stream we shift
+    root of our time to the Unix Epoch date (1970-01-01), thus our 0 is
+    actually dt = 719163.
+
+    So here is a simple formula how convert our epoch-based seconds to dt values
+        dt = (secs / 86400) + 719163
+    Where 719163 is an offset of Unix Epoch (1970-01-01) since Rata Die
+    (0001-01-01) in dates.
+
+    */
+    typedef int dt_t;
+
+    // dt_core.h
+    typedef enum {
+        DT_MON       = 1,
+        DT_MONDAY    = 1,
+        DT_TUE       = 2,
+        DT_TUESDAY   = 2,
+        DT_WED       = 3,
+        DT_WEDNESDAY = 3,
+        DT_THU       = 4,
+        DT_THURSDAY  = 4,
+        DT_FRI       = 5,
+        DT_FRIDAY    = 5,
+        DT_SAT       = 6,
+        DT_SATURDAY  = 6,
+        DT_SUN       = 7,
+        DT_SUNDAY    = 7,
+    } dt_dow_t;
+
+    dt_t     dt_from_rdn     (int n);
+    dt_t     dt_from_yd      (int y, int d);
+    dt_t     dt_from_ymd     (int y, int m, int d);
+    dt_t     dt_from_yqd     (int y, int q, int d);
+    dt_t     dt_from_ywd     (int y, int w, int d);
+
+    void     dt_to_yd        (dt_t dt, int *y, int *d);
+    void     dt_to_ymd       (dt_t dt, int *y, int *m, int *d);
+    void     dt_to_yqd       (dt_t dt, int *y, int *q, int *d);
+    void     dt_to_ywd       (dt_t dt, int *y, int *w, int *d);
+
+    int      dt_rdn          (dt_t dt);
+    dt_dow_t dt_dow          (dt_t dt);
+
+    // dt_parse_iso.h
+    size_t dt_parse_iso_date          (const char *str, size_t len, dt_t *dt);
+
+    size_t dt_parse_iso_time          (const char *str, size_t len, int *sod, int *nsec);
+    size_t dt_parse_iso_time_basic    (const char *str, size_t len, int *sod, int *nsec);
+    size_t dt_parse_iso_time_extended (const char *str, size_t len, int *sod, int *nsec);
+
+    size_t dt_parse_iso_zone          (const char *str, size_t len, int *offset);
+    size_t dt_parse_iso_zone_basic    (const char *str, size_t len, int *offset);
+    size_t dt_parse_iso_zone_extended (const char *str, size_t len, int *offset);
+    size_t dt_parse_iso_zone_lenient  (const char *str, size_t len, int *offset);
+
+    // dt_tm.h
+    dt_t    dt_from_struct_tm  (const struct tm *tm);
+    void    dt_to_struct_tm    (dt_t dt, struct tm *tm);
+
+    // <asm-generic/posix_types.h>
+    typedef long            __kernel_long_t;
+    typedef unsigned long   __kernel_ulong_t;
+    // /usr/include/x86_64-linux-gnu/bits/types/time_t.h
+    typedef long            time_t;
+
+
+    // <time.h>
+    typedef __kernel_long_t	__kernel_time_t;
+    typedef __kernel_long_t	__kernel_suseconds_t;
+
+    struct timespec {
+        __kernel_time_t	        tv_sec;     /* seconds */
+        long                    tv_nsec;    /* nanoseconds */
+    };
+
+    struct timeval {
+        __kernel_time_t	        tv_sec;	    /* seconds */
+        __kernel_suseconds_t    tv_usec;    /* microseconds */
+    };
+
+    struct timezone {
+        int	tz_minuteswest;     /* minutes west of Greenwich */
+        int	tz_dsttime;	        /* type of dst correction */
+    };
+
+    // /usr/include/x86_64-linux-gnu/sys/time.h
+    typedef struct timezone * __timezone_ptr_t;
+
+    /* Get the current time of day and timezone information,
+       putting it into *TV and *TZ.  If TZ is NULL, *TZ is not filled.
+       Returns 0 on success, -1 on errors.
+
+       NOTE: This form of timezone information is obsolete.
+       Use the functions and variables declared in <time.h> instead.  */
+    int gettimeofday (struct timeval *__tv, struct timezone * __tz);
+
+    // /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h
+    /* ISO C `broken-down time' structure.  */
+    struct tm
+    {
+        int tm_sec;	        /* Seconds.	[0-60] (1 leap second) */
+        int tm_min;	        /* Minutes.	[0-59] */
+        int tm_hour;        /* Hours.	[0-23] */
+        int tm_mday;        /* Day.		[1-31] */
+        int tm_mon;	        /* Month.	[0-11] */
+        int tm_year;        /* Year	- 1900.  */
+        int tm_wday;        /* Day of week.	[0-6] */
+        int tm_yday;        /* Days in year.[0-365]	*/
+        int tm_isdst;       /* DST.		[-1/0/1]*/
+
+        long int tm_gmtoff; /* Seconds east of UTC.  */
+        const char *tm_zone;/* Timezone abbreviation.  */
+    };
+
+    // <time.h>
+    /* Return the current time and put it in *TIMER if TIMER is not NULL.  */
+    time_t time (time_t *__timer);
+
+    /* Format TP into S according to FORMAT.
+    Write no more than MAXSIZE characters and return the number
+    of characters written, or 0 if it would exceed MAXSIZE.  */
+    size_t strftime (char * __s, size_t __maxsize, const char * __format,
+                     const struct tm * __tp);
+
+    /* Parse S according to FORMAT and store binary time information in TP.
+    The return value is a pointer to the first unparsed character in S.  */
+    char *strptime (const char * __s, const char * __fmt, struct tm *__tp);
+
+    /* Return the `struct tm' representation of *TIMER in UTC,
+    using *TP to store the result.  */
+    struct tm *gmtime_r (const time_t * __timer, struct tm * __tp);
+
+    /* Return the `struct tm' representation of *TIMER in local time,
+    using *TP to store the result.  */
+    struct tm *localtime_r (const time_t * __timer, struct tm * __tp);
+
+    /* Return a string of the form "Day Mon dd hh:mm:ss yyyy\n"
+    that is the representation of TP in this format.  */
+    char *asctime (const struct tm *__tp);
+
+    /* Equivalent to `asctime (localtime (timer))'.  */
+    char *ctime (const time_t *__timer);
+
+]]
+
+local builtin = ffi.C
+
+local SECS_PER_DAY     = 86400
+local NANOS_PER_SEC    = 1000000000LL
+
+-- c-dt/dt_config.h
+
+-- Unix, January 1, 1970, Thursday
+local DT_EPOCH_1970_OFFSET = 719163LL
+
+
+local datetime_t = ffi.typeof('struct datetime_t')
+local interval_t = ffi.typeof('struct datetime_interval_t')
+
+local function is_interval(o)
+    return type(o) == 'cdata' and ffi.istype(interval_t, o)
+end
+
+local function is_datetime(o)
+    return type(o) == 'cdata' and ffi.istype(datetime_t, o)
+end
+
+local function is_date_interval(o)
+    return is_datetime(o) or is_interval(o)
+end
+
+local function interval_new()
+    local interval = ffi.new(interval_t)
+    return interval
+end
+
+local function check_date(o, message, lvl)
+    if lvl == nil then
+        lvl = 2
+    end
+    if not is_datetime(o) then
+        return error(("%s: expected datetime, but received %s"):
+                     format(message, o), lvl)
+    end
+end
+
+local function check_date_interval(o, message, lvl)
+    if lvl == nil then
+        lvl = 2
+    end
+    if not (is_datetime(o) or is_interval(o)) then
+        return error(("%s: expected datetime or interval, but received %s"):
+                     format(message, o), lvl)
+    end
+end
+
+local function check_interval(o, message, lvl)
+    if lvl == nil then
+        lvl = 2
+    end
+    if not is_interval(o) then
+        return error(("%s: expected interval, but received %s"):
+                     format(message, o), lvl)
+    end
+end
+
+local function check_str(s, message, lvl)
+    if lvl == nil then
+        lvl = 2
+    end
+    if not type(s) == 'string' then
+        return error(("%s: expected string, but received %s"):
+                     format(message, s), lvl)
+    end
+end
+
+local function datetime_cmp(lhs, rhs)
+    if not is_date_interval(lhs) or
+       not is_date_interval(rhs) then
+       return nil
+    end
+    local sdiff = lhs.secs - rhs.secs
+    return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
+end
+
+local function datetime_eq(lhs, rhs)
+    local rc = datetime_cmp(lhs, rhs)
+    return rc ~= nil and rc == 0 or false
+end
+
+local function datetime_lt(lhs, rhs)
+    local rc = datetime_cmp(lhs, rhs)
+    return rc ~= nil and rc < 0 or false
+end
+
+local function datetime_le(lhs, rhs)
+    local rc = datetime_cmp(lhs, rhs)
+    return rc ~= nil and rc <= 0 or false
+end
+
+local function datetime_serialize(self)
+    -- Allow YAML, MsgPack and JSON to dump objects with sockets
+    return { secs = self.secs, nsec = self.nsec, offset = self.offset }
+end
+
+local function interval_serialize(self)
+    -- Allow YAML and JSON to dump objects with sockets
+    return { secs = self.secs, nsec = self.nsec }
+end
+
+local function local_rd(o)
+    return math.floor(tonumber(o.secs / SECS_PER_DAY)) + DT_EPOCH_1970_OFFSET
+end
+
+local function local_dt(o)
+    return builtin.dt_from_rdn(local_rd(o))
+end
+
+local function _normalize_nsec(secs, nsec)
+    if nsec < 0 then
+        secs = secs - 1
+        nsec = nsec + NANOS_PER_SEC
+    elseif nsec >= NANOS_PER_SEC then
+        secs = secs + 1
+        nsec = nsec - NANOS_PER_SEC
+    end
+    return secs, nsec
+end
+
+local function check_range(v, range, txt)
+    assert(#range == 2)
+    if not (v >= range[1] and v <= range[2]) then
+        error(('value %d of %s is out of allowed range [%d, %d]'):
+              format(v, txt, range[1], range[2]))
+    end
+end
+
+local datetime_index_handlers = {
+    unixtime = function(self)
+        return self.secs
+    end,
+
+    timestamp = function(self)
+        return tonumber(self.secs) + self.nsec / 1e9
+    end,
+
+    nanoseconds = function(self)
+        return self.secs * 1e9 + self.nsec
+    end,
+
+    microseconds = function(self)
+        return self.secs * 1e6 + self.nsec / 1e3
+    end,
+
+    milliseconds = function(self)
+        return self.secs * 1e3 + self.nsec / 1e6
+    end,
+
+    seconds = function(self)
+        return tonumber(self.secs) + self.nsec / 1e9
+    end,
+
+    minutes = function(self)
+        return (tonumber(self.secs) + self.nsec / 1e9) / 60
+    end,
+
+    hours = function(self)
+        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60)
+    end,
+
+    days = function(self)
+        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60) / 24
+    end,
+}
+
+local datetime_index = function(self, key)
+    return datetime_index_handlers[key] ~= nil and
+           datetime_index_handlers[key](self) or nil
+end
+
+local datetime_newindex_handlers = {
+    unixtime = function(self, value)
+        self.secs = value
+        self.nsec, self.offset = 0, 0
+    end,
+
+    timestamp = function(self, value)
+        local secs, frac = math.modf(value)
+        self.secs = secs
+        self.nsec = frac * 1e9
+        self.offset = 0
+    end,
+}
+
+local function datetime_newindex(self, key, value)
+    if datetime_newindex_handlers[key] ~= nil then
+        datetime_newindex_handlers[key](self, value)
+    end
+end
+
+local function datetime_new_raw(secs, nsec, offset)
+    local dt_obj = ffi.new(datetime_t)
+    dt_obj.secs = secs
+    dt_obj.nsec = nsec
+    dt_obj.offset = offset
+    return dt_obj
+end
+
+local function mk_timestamp(dt, sp, fp, offset)
+    local epochV = dt ~= nil and (builtin.dt_rdn(dt) - DT_EPOCH_1970_OFFSET) *
+                   SECS_PER_DAY or 0
+    local spV = sp ~= nil and sp or 0
+    local fpV = fp ~= nil and fp or 0
+    local ofsV = offset ~= nil and offset or 0
+    return datetime_new_raw (epochV + spV - ofsV * 60, fpV, ofsV)
+end
+
+-- create @datetime_t given object @o fields
+local function datetime_new(o)
+    if o == nil then
+        return datetime_new_raw(0, 0, 0)
+    end
+    local secs = 0
+    local nsec = 0
+    local offset = 0
+    local easy_way = false
+    local y = 0
+    local M = 0
+    local d = 0
+    local ymd = false
+
+    local h = 0
+    local m = 0
+    local s = 0
+    local frac = 0
+    local hms = false
+
+    local dt = 0
+
+    local handlers = {
+        secs = function(_, v)
+            secs = v
+            easy_way = true
+        end,
+
+        nsec = function(_, v)
+            nsec = v
+            easy_way = true
+        end,
+
+        offset = function (_, v)
+            offset = v
+            easy_way = true
+        end,
+
+        year = function(k, v)
+            check_range(v, {1, 9999}, k)
+            y = v
+            ymd = true
+        end,
+
+        month = function(k, v)
+            check_range(v, {1, 12}, k)
+            M = v
+            ymd = true
+        end,
+
+        day = function(k, v)
+            check_range(v, {1, 31}, k)
+            d = v
+            ymd = true
+        end,
+
+        hour = function(k, v)
+            check_range(v, {0, 23}, k)
+            h = v
+            hms = true
+        end,
+
+        minute = function(k, v)
+            check_range(v, {0, 59}, k)
+            m = v
+            hms = true
+        end,
+
+        second = function(k, v)
+            check_range(v, {0, 60}, k)
+            s, frac = math.modf(v)
+            frac = frac * 1e9 -- convert fraction to nanoseconds
+            hms = true
+        end,
+
+        -- tz offset in minutes
+        tz = function(k, v)
+            check_range(v, {0, 720}, k)
+            offset = v
+        end
+    }
+    for key, value in pairs(o) do
+        handlers[key](key, value)
+    end
+
+    -- .sec, .nsec, .offset
+    if easy_way then
+        return datetime_new_raw(secs, nsec, offset)
+    end
+
+    -- .year, .month, .day
+    if ymd then
+        dt = dt + builtin.dt_from_ymd(y, M, d)
+    end
+
+    -- .hour, .minute, .second
+    if hms then
+        secs = h * 3600 + m * 60 + s
+    end
+
+    return mk_timestamp(dt, secs, frac, offset)
+end
+
+local function date_first(lhs, rhs)
+    if is_datetime(lhs) then
+        return lhs, rhs
+    else
+        return rhs, lhs
+    end
+end
+
+local function error_incompatible(name)
+    error(("datetime:%s() - incompatible type of arguments"):
+          format(name), 3)
+end
+
+local function datetime_sub(lhs, rhs)
+    check_date_interval(lhs, "operator -")
+    local d, s = lhs, rhs
+    local left_t = ffi.typeof(d)
+    local right_t = ffi.typeof(s)
+    local o
+
+    if left_t == datetime_t then
+        -- left is date, right is date or generic interval
+        if (right_t == datetime_t or right_t == interval_t) then
+            o = right_t == datetime_t and interval_new() or datetime_new()
+            o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
+                                            lhs.nsec - rhs.nsec)
+            return o
+        else
+            error_incompatible("operator -")
+        end
+    -- both left and right are generic intervals
+    elseif left_t == interval_t and right_t == interval_t then
+        o = interval_new()
+        o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
+                                        lhs.nsec - rhs.nsec)
+        return o
+    else
+        error_incompatible("operator -")
+    end
+end
+
+local function datetime_add(lhs, rhs)
+    local d, s = date_first(lhs, rhs)
+
+    check_date_interval(d, "operator +")
+    check_interval(s, "operator +")
+    local left_t = ffi.typeof(d)
+    local right_t = ffi.typeof(s)
+    local o
+
+    -- left is date, right is date or interval
+    if left_t == datetime_t and right_t == interval_t then
+        o = datetime_new()
+        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
+        return o
+    -- both left and right are generic intervals
+    elseif left_t == interval_t and right_t == interval_t then
+        o = interval_new()
+        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
+        return o
+    else
+        error_incompatible("operator +")
+    end
+end
+
+-- simple parse functions:
+-- parse_date/parse_time/parse_zone
+
+--[[
+    Basic      Extended
+    20121224   2012-12-24   Calendar date   (ISO 8601)
+    2012359    2012-359     Ordinal date    (ISO 8601)
+    2012W521   2012-W52-1   Week date       (ISO 8601)
+    2012Q485   2012-Q4-85   Quarter date
+]]
+
+local function parse_date(str)
+    check_str("datetime.parse_date()")
+    local dt = ffi.new('dt_t[1]')
+    local len = builtin.dt_parse_iso_date(str, #str, dt)
+    return len > 0 and mk_timestamp(dt[0]) or nil, tonumber(len)
+end
+
+--[[
+    Basic               Extended
+    T12                 N/A
+    T1230               T12:30
+    T123045             T12:30:45
+    T123045.123456789   T12:30:45.123456789
+    T123045,123456789   T12:30:45,123456789
+
+    The time designator [T] may be omitted.
+]]
+local function parse_time(str)
+    check_str("datetime.parse_time()")
+    local sp = ffi.new('int[1]')
+    local fp = ffi.new('int[1]')
+    local len = builtin.dt_parse_iso_time(str, #str, sp, fp)
+    return len > 0 and mk_timestamp(nil, sp[0], fp[0]) or nil,
+           tonumber(len)
+end
+
+--[[
+    Basic    Extended
+    Z        N/A
+    +hh      N/A
+    -hh      N/A
+    +hhmm    +hh:mm
+    -hhmm    -hh:mm
+]]
+local function parse_zone(str)
+    check_str("datetime.parse_zone()")
+    local offset = ffi.new('int[1]')
+    local len = builtin.dt_parse_iso_zone_lenient(str, #str, offset)
+    return len > 0 and mk_timestamp(nil, nil, nil, offset[0]) or nil,
+           tonumber(len)
+end
+
+
+--[[
+    aggregated parse functions
+    assumes to deal with date T time time_zone
+    at once
+
+    date [T] time [ ] time_zone
+]]
+local function parse(str)
+    check_str("datetime.parse()")
+    local dt = ffi.new('dt_t[1]')
+    local len = #str
+    local n = builtin.dt_parse_iso_date(str, len, dt)
+    local dt_ = dt[0]
+    if n == 0 or len == n then
+        return mk_timestamp(dt_)
+    end
+
+    str = str:sub(tonumber(n) + 1)
+
+    local ch = str:sub(1,1)
+    if ch:match('[Tt ]') == nil then
+        return mk_timestamp(dt_)
+    end
+
+    str = str:sub(2)
+    len = #str
+
+    local sp = ffi.new('int[1]')
+    local fp = ffi.new('int[1]')
+    local n = builtin.dt_parse_iso_time(str, len, sp, fp)
+    if n == 0 then
+        return mk_timestamp(dt_)
+    end
+    local sp_ = sp[0]
+    local fp_ = fp[0]
+    if len == n then
+        return mk_timestamp(dt_, sp_, fp_)
+    end
+
+    str = str:sub(tonumber(n) + 1)
+
+    if str:sub(1,1) == ' ' then
+        str = str:sub(2)
+    end
+
+    len = #str
+
+    local offset = ffi.new('int[1]')
+    n = builtin.dt_parse_iso_zone_lenient(str, len, offset)
+    if n == 0 then
+        return mk_timestamp(dt_, sp_, fp_)
+    end
+    return mk_timestamp(dt_, sp_, fp_, offset[0])
+end
+
+local function datetime_from(o)
+    if o == nil or type(o) == 'table' then
+        return datetime_new(o)
+    elseif type(o) == 'string' then
+        return parse(o)
+    end
+end
+
+local function local_now()
+    local p_tv = ffi.new('struct timeval [1]')
+    local rc = builtin.gettimeofday(p_tv, nil)
+    assert(rc == 0)
+
+    local secs = p_tv[0].tv_sec
+    local nsec = p_tv[0].tv_usec * 1000
+
+    local p_time = ffi.new('time_t[1]')
+    local p_tm = ffi.new('struct tm[1]')
+    builtin.time(p_time)
+    builtin.localtime_r(p_time, p_tm)
+    local ofs = p_tm[0].tm_gmtoff / 60 -- convert seconds to minutes
+
+    return datetime_new_raw(secs, nsec, ofs)
+end
+
+local function datetime_to_tm_ptr(o)
+    assert(is_datetime(o))
+    local p_tm = ffi.new('struct tm[1]')
+    -- dt_to_struct_tm() fills only date data
+    builtin.dt_to_struct_tm(local_dt(o), p_tm)
+
+    -- calculate the smaller data (hour, minute,
+    -- seconds) using datetime seconds value
+    local seconds_of_day = o.secs % 86400
+    local hour = (seconds_of_day / 3600) % 24
+    local minute = (seconds_of_day / 60) % 60
+    p_tm[0].tm_sec = seconds_of_day % 60
+    p_tm[0].tm_min = minute
+    p_tm[0].tm_hour = hour
+
+    p_tm[0].tm_gmtoff = o.offset * 60
+
+    return p_tm
+end
+
+local function asctime(o)
+    check_date(o, "datetime:asctime()")
+
+    local p_tm = datetime_to_tm_ptr(o)
+    return ffi.string(builtin.asctime(p_tm))
+end
+
+local function ctime(o)
+    check_date(o, "datetime:ctime()")
+    local p_time = ffi.new('time_t[1]')
+    p_time[0] = o.secs
+    return ffi.string(builtin.ctime(p_time))
+end
+
+local function strftime(fmt, o)
+    check_date(o, "datetime.strftime()")
+    local p_tm = datetime_to_tm_ptr(o)
+    local sz = builtin.strftime(nil, 1024, fmt, p_tm) + 1
+    local buff = ffi.new('char[?]', sz)
+    builtin.strftime(buff, sz, fmt, p_tm)
+    return ffi.string(buff)
+end
+
+local datetime_mt = {
+    __serialize = datetime_serialize,
+    __eq = datetime_eq,
+    __lt = datetime_lt,
+    __le = datetime_le,
+    __sub = datetime_sub,
+    __add = datetime_add,
+    __index = datetime_index,
+    __newindex = datetime_newindex,
+}
+
+local interval_mt = {
+    __serialize = interval_serialize,
+    __eq = datetime_eq,
+    __lt = datetime_lt,
+    __le = datetime_le,
+    __sub = datetime_sub,
+    __add = datetime_add,
+    __index = datetime_index,
+}
+
+ffi.metatype(interval_t, interval_mt)
+ffi.metatype(datetime_t, datetime_mt)
+
+return setmetatable(
+    {
+        new         = datetime_new,
+        interval    = interval_new,
+
+        parse       = parse,
+        parse_date  = parse_date,
+        parse_time  = parse_time,
+        parse_zone  = parse_zone,
+
+        now         = local_now,
+        strftime    = strftime,
+        asctime     = asctime,
+        ctime       = ctime,
+
+        is_datetime = is_datetime,
+        is_interval = is_interval,
+    }, {
+        __call = function(self, ...) return datetime_from(...) end
+    }
+)
diff --git a/src/lua/init.c b/src/lua/init.c
index f9738025d..127e935d7 100644
--- a/src/lua/init.c
+++ b/src/lua/init.c
@@ -129,7 +129,8 @@ extern char strict_lua[],
 	parse_lua[],
 	process_lua[],
 	humanize_lua[],
-	memprof_lua[]
+	memprof_lua[],
+	datetime_lua[]
 ;
 
 static const char *lua_modules[] = {
@@ -184,6 +185,7 @@ static const char *lua_modules[] = {
 	"memprof.process", process_lua,
 	"memprof.humanize", humanize_lua,
 	"memprof", memprof_lua,
+	"datetime", datetime_lua,
 	NULL
 };
 
diff --git a/src/lua/utils.c b/src/lua/utils.c
index ebc19206c..611044b6f 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -47,6 +47,9 @@ static uint32_t CTID_STRUCT_IBUF_PTR;
 uint32_t CTID_CHAR_PTR;
 uint32_t CTID_CONST_CHAR_PTR;
 uint32_t CTID_UUID;
+uint32_t CTID_DATETIME = 0;
+uint32_t CTID_INTERVAL = 0;
+
 
 void *
 luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
@@ -107,6 +110,12 @@ luaL_pushuuid(struct lua_State *L)
 	return luaL_pushcdata(L, CTID_UUID);
 }
 
+struct datetime_t *
+luaL_pushdatetime(struct lua_State *L)
+{
+	return luaL_pushcdata(L, CTID_DATETIME);
+}
+
 int
 luaL_iscdata(struct lua_State *L, int idx)
 {
@@ -712,6 +721,24 @@ tarantool_lua_utils_init(struct lua_State *L)
 	CTID_UUID = luaL_ctypeid(L, "struct tt_uuid");
 	assert(CTID_UUID != 0);
 
+	rc = luaL_cdef(L, "struct datetime_t {"
+			  "int64_t secs;"
+			  "int32_t nsec;"
+			  "int32_t offset;"
+			  "};");
+	assert(rc == 0);
+	(void) rc;
+	CTID_DATETIME = luaL_ctypeid(L, "struct datetime_t");
+	assert(CTID_DATETIME != 0);
+	rc = luaL_cdef(L, "struct datetime_interval_t {"
+			  "int64_t secs;"
+			  "int32_t nsec;"
+			  "};");
+	assert(rc == 0);
+	(void) rc;
+	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval_t");
+	assert(CTID_INTERVAL != 0);
+
 	lua_pushcfunction(L, luaT_newthread_wrapper);
 	luaT_newthread_ref = luaL_ref(L, LUA_REGISTRYINDEX);
 	return 0;
diff --git a/src/lua/utils.h b/src/lua/utils.h
index 969edca45..bf56b1bb5 100644
--- a/src/lua/utils.h
+++ b/src/lua/utils.h
@@ -59,6 +59,7 @@ struct lua_State;
 struct ibuf;
 typedef struct ibuf box_ibuf_t;
 struct tt_uuid;
+struct datetime_t;
 
 /**
  * Single global lua_State shared by core and modules.
@@ -71,10 +72,21 @@ extern struct lua_State *tarantool_L;
 extern uint32_t CTID_CHAR_PTR;
 extern uint32_t CTID_CONST_CHAR_PTR;
 extern uint32_t CTID_UUID;
+extern uint32_t CTID_DATETIME;
+extern uint32_t CTID_INTERVAL;
 
 struct tt_uuid *
 luaL_pushuuid(struct lua_State *L);
 
+/**
+ * @brief Push cdata of a datetime_t type onto the stack.
+ * @param L Lua State
+ * @sa luaL_pushcdata
+ * @return memory associated with this datetime_t data
+ */
+struct datetime_t *
+luaL_pushdatetime(struct lua_State *L);
+
 /** \cond public */
 
 /**
diff --git a/test/unit/CMakeLists.txt b/test/unit/CMakeLists.txt
index 5bb7cd6e7..f8320aebd 100644
--- a/test/unit/CMakeLists.txt
+++ b/test/unit/CMakeLists.txt
@@ -56,6 +56,8 @@ add_executable(uuid.test uuid.c core_test_utils.c)
 target_link_libraries(uuid.test uuid unit)
 add_executable(random.test random.c core_test_utils.c)
 target_link_libraries(random.test core unit)
+add_executable(datetime.test datetime.c)
+target_link_libraries(datetime.test cdt unit)
 
 add_executable(bps_tree.test bps_tree.cc)
 target_link_libraries(bps_tree.test small misc)
diff --git a/test/unit/datetime.c b/test/unit/datetime.c
new file mode 100644
index 000000000..b6f568c03
--- /dev/null
+++ b/test/unit/datetime.c
@@ -0,0 +1,221 @@
+#include "dt.h"
+#include <assert.h>
+#include <stdint.h>
+#include <string.h>
+#include <time.h>
+
+#include "unit.h"
+
+const char sample[] = "2012-12-24T15:30Z";
+
+#define S(s) {s, sizeof(s) - 1}
+struct {
+	const char * sz;
+	size_t len;
+} tests[] = {
+	S("2012-12-24 15:30Z"),
+	S("2012-12-24 15:30z"),
+	S("2012-12-24 15:30"),
+	S("2012-12-24 16:30+01:00"),
+	S("2012-12-24 16:30+0100"),
+	S("2012-12-24 16:30+01"),
+	S("2012-12-24 14:30-01:00"),
+	S("2012-12-24 14:30-0100"),
+	S("2012-12-24 14:30-01"),
+	S("2012-12-24 15:30:00Z"),
+	S("2012-12-24 15:30:00z"),
+	S("2012-12-24 15:30:00"),
+	S("2012-12-24 16:30:00+01:00"),
+	S("2012-12-24 16:30:00+0100"),
+	S("2012-12-24 14:30:00-01:00"),
+	S("2012-12-24 14:30:00-0100"),
+	S("2012-12-24 15:30:00.123456Z"),
+	S("2012-12-24 15:30:00.123456z"),
+	S("2012-12-24 15:30:00.123456"),
+	S("2012-12-24 16:30:00.123456+01:00"),
+	S("2012-12-24 16:30:00.123456+01"),
+	S("2012-12-24 14:30:00.123456-01:00"),
+	S("2012-12-24 14:30:00.123456-01"),
+	S("2012-12-24t15:30Z"),
+	S("2012-12-24t15:30z"),
+	S("2012-12-24t15:30"),
+	S("2012-12-24t16:30+01:00"),
+	S("2012-12-24t16:30+0100"),
+	S("2012-12-24t14:30-01:00"),
+	S("2012-12-24t14:30-0100"),
+	S("2012-12-24t15:30:00Z"),
+	S("2012-12-24t15:30:00z"),
+	S("2012-12-24t15:30:00"),
+	S("2012-12-24t16:30:00+01:00"),
+	S("2012-12-24t16:30:00+0100"),
+	S("2012-12-24t14:30:00-01:00"),
+	S("2012-12-24t14:30:00-0100"),
+	S("2012-12-24t15:30:00.123456Z"),
+	S("2012-12-24t15:30:00.123456z"),
+	S("2012-12-24t16:30:00.123456+01:00"),
+	S("2012-12-24t14:30:00.123456-01:00"),
+	S("2012-12-24 16:30 +01:00"),
+	S("2012-12-24 14:30 -01:00"),
+	S("2012-12-24 15:30 UTC"),
+	S("2012-12-24 16:30 UTC+1"),
+	S("2012-12-24 16:30 UTC+01"),
+	S("2012-12-24 16:30 UTC+0100"),
+	S("2012-12-24 16:30 UTC+01:00"),
+	S("2012-12-24 14:30 UTC-1"),
+	S("2012-12-24 14:30 UTC-01"),
+	S("2012-12-24 14:30 UTC-01:00"),
+	S("2012-12-24 14:30 UTC-0100"),
+	S("2012-12-24 15:30 GMT"),
+	S("2012-12-24 16:30 GMT+1"),
+	S("2012-12-24 16:30 GMT+01"),
+	S("2012-12-24 16:30 GMT+0100"),
+	S("2012-12-24 16:30 GMT+01:00"),
+	S("2012-12-24 14:30 GMT-1"),
+	S("2012-12-24 14:30 GMT-01"),
+	S("2012-12-24 14:30 GMT-01:00"),
+	S("2012-12-24 14:30 GMT-0100"),
+	S("2012-12-24 14:30 -01:00"),
+	S("2012-12-24 16:30:00 +01:00"),
+	S("2012-12-24 14:30:00 -01:00"),
+	S("2012-12-24 16:30:00.123456 +01:00"),
+	S("2012-12-24 14:30:00.123456 -01:00"),
+	S("2012-12-24 15:30:00.123456 -00:00"),
+	S("20121224T1630+01:00"),
+	S("2012-12-24T1630+01:00"),
+	S("20121224T16:30+01"),
+	S("20121224T16:30 +01"),
+};
+#undef S
+
+#define DIM(a) (sizeof(a) / sizeof(a[0]))
+
+// p5-time-moment/src/moment_parse.c: parse_string_lenient()
+static int
+parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
+	       int32_t *op)
+{
+	size_t n;
+	dt_t dt;
+	char c;
+	int sod = 0, nanosecond = 0, offset = 0;
+
+	n = dt_parse_iso_date(str, len, &dt);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	c = str[n++];
+	if (!(c == 'T' || c == 't' || c == ' '))
+		return 1;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_time(str, len, &sod, &nanosecond);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	if (str[n] == ' ')
+	n++;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_zone_lenient(str, len, &offset);
+	if (!n || n != len)
+		return 1;
+
+exit:
+	*sp = ((int64_t)dt_rdn(dt) - 719163) * 86400 + sod - offset * 60;
+	*np = nanosecond;
+	*op = offset;
+
+	return 0;
+}
+
+// avoid introducing external datetime.h dependency
+// - just copy paste it for today
+#define SECS_PER_DAY      86400
+#define NANOS_PER_SEC     1000000000
+#define DT_EPOCH_1970_OFFSET 719163
+
+
+struct datetime_t {
+	int64_t secs;
+	int32_t nsec;
+	int32_t offset;
+};
+
+static int
+local_rd(const struct datetime_t * dt) {
+	return (int)(dt->secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET;
+}
+
+static int
+local_dt(const struct datetime_t * dt) {
+	return dt_from_rdn(local_rd(dt));
+}
+
+
+struct tm*
+datetime_to_tm(struct datetime_t * dt)
+{
+	static struct tm tm;
+
+	memset(&tm, 0, sizeof(tm));
+	dt_to_struct_tm(local_dt(dt), &tm);
+
+	int seconds_of_day = dt->secs % 86400;
+	tm.tm_hour = (seconds_of_day / 3600) % 24;
+	tm.tm_min = (seconds_of_day / 60) % 60;
+	tm.tm_sec = seconds_of_day % 60;
+
+	return &tm;
+}
+
+static void datetime_test(void)
+{
+	size_t index;
+	int64_t secs_expected;
+	int32_t nanosecs;
+	int32_t ofs;
+
+	plan(355);
+	parse_datetime(sample, sizeof(sample) - 1,
+		       &secs_expected, &nanosecs, &ofs);
+
+	for (index = 0; index < DIM(tests); index++) {
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+						&secs, &nanosecs, &ofs);
+		is(rc, 0, "correct parse_datetime return value for '%s'",
+		   tests[index].sz);
+		is(secs, secs_expected, "correct parse_datetime output "
+		   "seconds for '%s", tests[index].sz);
+
+		// check that stringized literal produces the same date
+		// time fields
+		static char buff[40];
+		struct datetime_t dt = {secs, nanosecs, ofs};
+		// datetime_to_tm returns time in GMT zone
+		struct tm * p_tm = datetime_to_tm(&dt);
+		size_t len = strftime(buff, sizeof buff, "%F %T%z", p_tm);
+		ok(len > 0, "strftime");
+		rc = parse_datetime(buff, len, &dt.secs, &dt.nsec, &dt.offset);
+		is(rc, 0, "correct parse_datetime return value for '%s'", buff);
+		is(secs, dt.secs,
+		   "reversible seconds via strftime for '%s", buff);
+	}
+}
+
+int
+main(void)
+{
+	plan(1);
+	datetime_test();
+
+	return check_plan();
+}
diff --git a/test/unit/datetime.result b/test/unit/datetime.result
new file mode 100644
index 000000000..f106fa769
--- /dev/null
+++ b/test/unit/datetime.result
@@ -0,0 +1,358 @@
+1..1
+    1..355
+    ok 1 - correct parse_datetime return value for '2012-12-24 15:30Z'
+    ok 2 - correct parse_datetime output seconds for '2012-12-24 15:30Z
+    ok 3 - strftime
+    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 6 - correct parse_datetime return value for '2012-12-24 15:30z'
+    ok 7 - correct parse_datetime output seconds for '2012-12-24 15:30z
+    ok 8 - strftime
+    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 11 - correct parse_datetime return value for '2012-12-24 15:30'
+    ok 12 - correct parse_datetime output seconds for '2012-12-24 15:30
+    ok 13 - strftime
+    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 16 - correct parse_datetime return value for '2012-12-24 16:30+01:00'
+    ok 17 - correct parse_datetime output seconds for '2012-12-24 16:30+01:00
+    ok 18 - strftime
+    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 21 - correct parse_datetime return value for '2012-12-24 16:30+0100'
+    ok 22 - correct parse_datetime output seconds for '2012-12-24 16:30+0100
+    ok 23 - strftime
+    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 26 - correct parse_datetime return value for '2012-12-24 16:30+01'
+    ok 27 - correct parse_datetime output seconds for '2012-12-24 16:30+01
+    ok 28 - strftime
+    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 31 - correct parse_datetime return value for '2012-12-24 14:30-01:00'
+    ok 32 - correct parse_datetime output seconds for '2012-12-24 14:30-01:00
+    ok 33 - strftime
+    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 36 - correct parse_datetime return value for '2012-12-24 14:30-0100'
+    ok 37 - correct parse_datetime output seconds for '2012-12-24 14:30-0100
+    ok 38 - strftime
+    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 41 - correct parse_datetime return value for '2012-12-24 14:30-01'
+    ok 42 - correct parse_datetime output seconds for '2012-12-24 14:30-01
+    ok 43 - strftime
+    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 46 - correct parse_datetime return value for '2012-12-24 15:30:00Z'
+    ok 47 - correct parse_datetime output seconds for '2012-12-24 15:30:00Z
+    ok 48 - strftime
+    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 51 - correct parse_datetime return value for '2012-12-24 15:30:00z'
+    ok 52 - correct parse_datetime output seconds for '2012-12-24 15:30:00z
+    ok 53 - strftime
+    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 56 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 57 - correct parse_datetime output seconds for '2012-12-24 15:30:00
+    ok 58 - strftime
+    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 61 - correct parse_datetime return value for '2012-12-24 16:30:00+01:00'
+    ok 62 - correct parse_datetime output seconds for '2012-12-24 16:30:00+01:00
+    ok 63 - strftime
+    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 66 - correct parse_datetime return value for '2012-12-24 16:30:00+0100'
+    ok 67 - correct parse_datetime output seconds for '2012-12-24 16:30:00+0100
+    ok 68 - strftime
+    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 71 - correct parse_datetime return value for '2012-12-24 14:30:00-01:00'
+    ok 72 - correct parse_datetime output seconds for '2012-12-24 14:30:00-01:00
+    ok 73 - strftime
+    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 76 - correct parse_datetime return value for '2012-12-24 14:30:00-0100'
+    ok 77 - correct parse_datetime output seconds for '2012-12-24 14:30:00-0100
+    ok 78 - strftime
+    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 81 - correct parse_datetime return value for '2012-12-24 15:30:00.123456Z'
+    ok 82 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456Z
+    ok 83 - strftime
+    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 86 - correct parse_datetime return value for '2012-12-24 15:30:00.123456z'
+    ok 87 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456z
+    ok 88 - strftime
+    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 91 - correct parse_datetime return value for '2012-12-24 15:30:00.123456'
+    ok 92 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456
+    ok 93 - strftime
+    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 96 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01:00'
+    ok 97 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01:00
+    ok 98 - strftime
+    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 101 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01'
+    ok 102 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01
+    ok 103 - strftime
+    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 106 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01:00'
+    ok 107 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01:00
+    ok 108 - strftime
+    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 111 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01'
+    ok 112 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01
+    ok 113 - strftime
+    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 116 - correct parse_datetime return value for '2012-12-24t15:30Z'
+    ok 117 - correct parse_datetime output seconds for '2012-12-24t15:30Z
+    ok 118 - strftime
+    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 121 - correct parse_datetime return value for '2012-12-24t15:30z'
+    ok 122 - correct parse_datetime output seconds for '2012-12-24t15:30z
+    ok 123 - strftime
+    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 126 - correct parse_datetime return value for '2012-12-24t15:30'
+    ok 127 - correct parse_datetime output seconds for '2012-12-24t15:30
+    ok 128 - strftime
+    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 131 - correct parse_datetime return value for '2012-12-24t16:30+01:00'
+    ok 132 - correct parse_datetime output seconds for '2012-12-24t16:30+01:00
+    ok 133 - strftime
+    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 136 - correct parse_datetime return value for '2012-12-24t16:30+0100'
+    ok 137 - correct parse_datetime output seconds for '2012-12-24t16:30+0100
+    ok 138 - strftime
+    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 141 - correct parse_datetime return value for '2012-12-24t14:30-01:00'
+    ok 142 - correct parse_datetime output seconds for '2012-12-24t14:30-01:00
+    ok 143 - strftime
+    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 146 - correct parse_datetime return value for '2012-12-24t14:30-0100'
+    ok 147 - correct parse_datetime output seconds for '2012-12-24t14:30-0100
+    ok 148 - strftime
+    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 151 - correct parse_datetime return value for '2012-12-24t15:30:00Z'
+    ok 152 - correct parse_datetime output seconds for '2012-12-24t15:30:00Z
+    ok 153 - strftime
+    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 156 - correct parse_datetime return value for '2012-12-24t15:30:00z'
+    ok 157 - correct parse_datetime output seconds for '2012-12-24t15:30:00z
+    ok 158 - strftime
+    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 161 - correct parse_datetime return value for '2012-12-24t15:30:00'
+    ok 162 - correct parse_datetime output seconds for '2012-12-24t15:30:00
+    ok 163 - strftime
+    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 166 - correct parse_datetime return value for '2012-12-24t16:30:00+01:00'
+    ok 167 - correct parse_datetime output seconds for '2012-12-24t16:30:00+01:00
+    ok 168 - strftime
+    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 171 - correct parse_datetime return value for '2012-12-24t16:30:00+0100'
+    ok 172 - correct parse_datetime output seconds for '2012-12-24t16:30:00+0100
+    ok 173 - strftime
+    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 176 - correct parse_datetime return value for '2012-12-24t14:30:00-01:00'
+    ok 177 - correct parse_datetime output seconds for '2012-12-24t14:30:00-01:00
+    ok 178 - strftime
+    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 181 - correct parse_datetime return value for '2012-12-24t14:30:00-0100'
+    ok 182 - correct parse_datetime output seconds for '2012-12-24t14:30:00-0100
+    ok 183 - strftime
+    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 186 - correct parse_datetime return value for '2012-12-24t15:30:00.123456Z'
+    ok 187 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456Z
+    ok 188 - strftime
+    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 191 - correct parse_datetime return value for '2012-12-24t15:30:00.123456z'
+    ok 192 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456z
+    ok 193 - strftime
+    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 196 - correct parse_datetime return value for '2012-12-24t16:30:00.123456+01:00'
+    ok 197 - correct parse_datetime output seconds for '2012-12-24t16:30:00.123456+01:00
+    ok 198 - strftime
+    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 201 - correct parse_datetime return value for '2012-12-24t14:30:00.123456-01:00'
+    ok 202 - correct parse_datetime output seconds for '2012-12-24t14:30:00.123456-01:00
+    ok 203 - strftime
+    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 206 - correct parse_datetime return value for '2012-12-24 16:30 +01:00'
+    ok 207 - correct parse_datetime output seconds for '2012-12-24 16:30 +01:00
+    ok 208 - strftime
+    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 211 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
+    ok 212 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
+    ok 213 - strftime
+    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 216 - correct parse_datetime return value for '2012-12-24 15:30 UTC'
+    ok 217 - correct parse_datetime output seconds for '2012-12-24 15:30 UTC
+    ok 218 - strftime
+    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 221 - correct parse_datetime return value for '2012-12-24 16:30 UTC+1'
+    ok 222 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+1
+    ok 223 - strftime
+    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 226 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01'
+    ok 227 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01
+    ok 228 - strftime
+    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 231 - correct parse_datetime return value for '2012-12-24 16:30 UTC+0100'
+    ok 232 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+0100
+    ok 233 - strftime
+    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 236 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01:00'
+    ok 237 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01:00
+    ok 238 - strftime
+    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 241 - correct parse_datetime return value for '2012-12-24 14:30 UTC-1'
+    ok 242 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-1
+    ok 243 - strftime
+    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 246 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01'
+    ok 247 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01
+    ok 248 - strftime
+    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 251 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01:00'
+    ok 252 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01:00
+    ok 253 - strftime
+    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 256 - correct parse_datetime return value for '2012-12-24 14:30 UTC-0100'
+    ok 257 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-0100
+    ok 258 - strftime
+    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 261 - correct parse_datetime return value for '2012-12-24 15:30 GMT'
+    ok 262 - correct parse_datetime output seconds for '2012-12-24 15:30 GMT
+    ok 263 - strftime
+    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 266 - correct parse_datetime return value for '2012-12-24 16:30 GMT+1'
+    ok 267 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+1
+    ok 268 - strftime
+    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 271 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01'
+    ok 272 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01
+    ok 273 - strftime
+    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 276 - correct parse_datetime return value for '2012-12-24 16:30 GMT+0100'
+    ok 277 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+0100
+    ok 278 - strftime
+    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 281 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01:00'
+    ok 282 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01:00
+    ok 283 - strftime
+    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 286 - correct parse_datetime return value for '2012-12-24 14:30 GMT-1'
+    ok 287 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-1
+    ok 288 - strftime
+    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 291 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01'
+    ok 292 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01
+    ok 293 - strftime
+    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 296 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01:00'
+    ok 297 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01:00
+    ok 298 - strftime
+    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 301 - correct parse_datetime return value for '2012-12-24 14:30 GMT-0100'
+    ok 302 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-0100
+    ok 303 - strftime
+    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 306 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
+    ok 307 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
+    ok 308 - strftime
+    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 311 - correct parse_datetime return value for '2012-12-24 16:30:00 +01:00'
+    ok 312 - correct parse_datetime output seconds for '2012-12-24 16:30:00 +01:00
+    ok 313 - strftime
+    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 316 - correct parse_datetime return value for '2012-12-24 14:30:00 -01:00'
+    ok 317 - correct parse_datetime output seconds for '2012-12-24 14:30:00 -01:00
+    ok 318 - strftime
+    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 321 - correct parse_datetime return value for '2012-12-24 16:30:00.123456 +01:00'
+    ok 322 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456 +01:00
+    ok 323 - strftime
+    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 326 - correct parse_datetime return value for '2012-12-24 14:30:00.123456 -01:00'
+    ok 327 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456 -01:00
+    ok 328 - strftime
+    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 331 - correct parse_datetime return value for '2012-12-24 15:30:00.123456 -00:00'
+    ok 332 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456 -00:00
+    ok 333 - strftime
+    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 336 - correct parse_datetime return value for '20121224T1630+01:00'
+    ok 337 - correct parse_datetime output seconds for '20121224T1630+01:00
+    ok 338 - strftime
+    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 341 - correct parse_datetime return value for '2012-12-24T1630+01:00'
+    ok 342 - correct parse_datetime output seconds for '2012-12-24T1630+01:00
+    ok 343 - strftime
+    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 346 - correct parse_datetime return value for '20121224T16:30+01'
+    ok 347 - correct parse_datetime output seconds for '20121224T16:30+01
+    ok 348 - strftime
+    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 351 - correct parse_datetime return value for '20121224T16:30 +01'
+    ok 352 - correct parse_datetime output seconds for '20121224T16:30 +01
+    ok 353 - strftime
+    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
+    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+ok 1 - subtests
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 3/9] lua, datetime: datetime tests
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build Timur Safin via Tarantool-patches
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime Timur Safin via Tarantool-patches
@ 2021-08-02  0:40 ` Timur Safin via Tarantool-patches
  2021-08-06  0:25   ` Safin Timur via Tarantool-patches
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 4/9] lua, datetime: display datetime Timur Safin via Tarantool-patches
                   ` (8 subsequent siblings)
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:40 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* created app-tap test for new builtin module `datetime.lua`
* added case to check datetime string formatting using:
  - asctime (gmt time);
  - ctime (local TZ time);
  - strftime (using given format).

* added positive/negative checks to datetime test:
  - for that,in the datetime.lua, we have extended api of `parse_date()`,
    `parse_time()`, and `parse_time_zone()` so they return
    not only parsed object, but also a length of parsed substring;
  - which allows us to parse even _partially_ valid strings like
    "20121224 Foo bar".

* checks calculated attributes to date object, e.g.:
  - timestamp, seconds, microseconds, minute, or hours

Part of #5941
---
 test/app-tap/datetime.test.lua | 206 +++++++++++++++++++++++++++++++++
 1 file changed, 206 insertions(+)
 create mode 100755 test/app-tap/datetime.test.lua

diff --git a/test/app-tap/datetime.test.lua b/test/app-tap/datetime.test.lua
new file mode 100755
index 000000000..f7541567e
--- /dev/null
+++ b/test/app-tap/datetime.test.lua
@@ -0,0 +1,206 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local test = tap.test("errno")
+local date = require('datetime')
+local ffi = require('ffi')
+
+
+test:plan(6)
+
+test:test("Simple tests for parser", function(test)
+    test:plan(2)
+    test:ok(date("1970-01-01T01:00:00Z") ==
+            date {year=1970, month=1, day=1, hour=1, minute=0, second=0})
+    test:ok(date("1970-01-01T02:00:00+02:00") ==
+            date {year=1970, month=1, day=1, hour=2, minute=0, second=0, tz=120})
+end)
+
+test:test("Multiple tests for parser (with nanoseconds)", function(test)
+    test:plan(165)
+    -- borrowed from p5-time-moments/t/180_from_string.t
+    local tests =
+    {
+        { '1970-01-01T00:00:00Z',                       0,           0,    0 },
+        { '1970-01-01T02:00:00+02:00',                  0,           0,  120 },
+        { '1970-01-01T01:30:00+01:30',                  0,           0,   90 },
+        { '1970-01-01T01:00:00+01:00',                  0,           0,   60 },
+        { '1970-01-01T00:01:00+00:01',                  0,           0,    1 },
+        { '1970-01-01T00:00:00+00:00',                  0,           0,    0 },
+        { '1969-12-31T23:59:00-00:01',                  0,           0,   -1 },
+        { '1969-12-31T23:00:00-01:00',                  0,           0,  -60 },
+        { '1969-12-31T22:30:00-01:30',                  0,           0,  -90 },
+        { '1969-12-31T22:00:00-02:00',                  0,           0, -120 },
+        { '1970-01-01T00:00:00.123456789Z',             0,   123456789,    0 },
+        { '1970-01-01T00:00:00.12345678Z',              0,   123456780,    0 },
+        { '1970-01-01T00:00:00.1234567Z',               0,   123456700,    0 },
+        { '1970-01-01T00:00:00.123456Z',                0,   123456000,    0 },
+        { '1970-01-01T00:00:00.12345Z',                 0,   123450000,    0 },
+        { '1970-01-01T00:00:00.1234Z',                  0,   123400000,    0 },
+        { '1970-01-01T00:00:00.123Z',                   0,   123000000,    0 },
+        { '1970-01-01T00:00:00.12Z',                    0,   120000000,    0 },
+        { '1970-01-01T00:00:00.1Z',                     0,   100000000,    0 },
+        { '1970-01-01T00:00:00.01Z',                    0,    10000000,    0 },
+        { '1970-01-01T00:00:00.001Z',                   0,     1000000,    0 },
+        { '1970-01-01T00:00:00.0001Z',                  0,      100000,    0 },
+        { '1970-01-01T00:00:00.00001Z',                 0,       10000,    0 },
+        { '1970-01-01T00:00:00.000001Z',                0,        1000,    0 },
+        { '1970-01-01T00:00:00.0000001Z',               0,         100,    0 },
+        { '1970-01-01T00:00:00.00000001Z',              0,          10,    0 },
+        { '1970-01-01T00:00:00.000000001Z',             0,           1,    0 },
+        { '1970-01-01T00:00:00.000000009Z',             0,           9,    0 },
+        { '1970-01-01T00:00:00.00000009Z',              0,          90,    0 },
+        { '1970-01-01T00:00:00.0000009Z',               0,         900,    0 },
+        { '1970-01-01T00:00:00.000009Z',                0,        9000,    0 },
+        { '1970-01-01T00:00:00.00009Z',                 0,       90000,    0 },
+        { '1970-01-01T00:00:00.0009Z',                  0,      900000,    0 },
+        { '1970-01-01T00:00:00.009Z',                   0,     9000000,    0 },
+        { '1970-01-01T00:00:00.09Z',                    0,    90000000,    0 },
+        { '1970-01-01T00:00:00.9Z',                     0,   900000000,    0 },
+        { '1970-01-01T00:00:00.99Z',                    0,   990000000,    0 },
+        { '1970-01-01T00:00:00.999Z',                   0,   999000000,    0 },
+        { '1970-01-01T00:00:00.9999Z',                  0,   999900000,    0 },
+        { '1970-01-01T00:00:00.99999Z',                 0,   999990000,    0 },
+        { '1970-01-01T00:00:00.999999Z',                0,   999999000,    0 },
+        { '1970-01-01T00:00:00.9999999Z',               0,   999999900,    0 },
+        { '1970-01-01T00:00:00.99999999Z',              0,   999999990,    0 },
+        { '1970-01-01T00:00:00.999999999Z',             0,   999999999,    0 },
+        { '1970-01-01T00:00:00.0Z',                     0,           0,    0 },
+        { '1970-01-01T00:00:00.00Z',                    0,           0,    0 },
+        { '1970-01-01T00:00:00.000Z',                   0,           0,    0 },
+        { '1970-01-01T00:00:00.0000Z',                  0,           0,    0 },
+        { '1970-01-01T00:00:00.00000Z',                 0,           0,    0 },
+        { '1970-01-01T00:00:00.000000Z',                0,           0,    0 },
+        { '1970-01-01T00:00:00.0000000Z',               0,           0,    0 },
+        { '1970-01-01T00:00:00.00000000Z',              0,           0,    0 },
+        { '1970-01-01T00:00:00.000000000Z',             0,           0,    0 },
+        { '1973-11-29T21:33:09Z',               123456789,           0,    0 },
+        { '2013-10-28T17:51:56Z',              1382982716,           0,    0 },
+        -- { '9999-12-31T23:59:59Z',            253402300799,           0,    0 },
+    }
+    for _, value in ipairs(tests) do
+        local str, epoch, nsec, offset
+        str, epoch, nsec, offset = unpack(value)
+        local dt = date(str)
+        test:ok(dt.secs == epoch, ('%s: dt.secs == %d'):format(str, epoch))
+        test:ok(dt.nsec == nsec, ('%s: dt.nsec == %d'):format(str, nsec))
+        test:ok(dt.offset == offset, ('%s: dt.offset == %d'):format(str, offset))
+    end
+end)
+
+ffi.cdef [[
+    void tzset(void);
+]]
+
+test:test("Datetime string formatting", function(test)
+    test:plan(7)
+    local str = "1970-01-01"
+    local t = date(str)
+    test:ok(t.secs == 0, ('%s: t.secs == %d'):format(str, tonumber(t.secs)))
+    test:ok(t.nsec == 0, ('%s: t.nsec == %d'):format(str, t.nsec))
+    test:ok(t.offset == 0, ('%s: t.offset == %d'):format(str, t.offset))
+    test:ok(date.asctime(t) == 'Thu Jan  1 00:00:00 1970\n', ('%s: asctime'):format(str))
+    -- ctime() is local timezone dependent. To make sure that
+    -- test is deterministic we enforce timezone via TZ environment
+    -- manipulations and calling tzset()
+
+    -- redefine timezone to be always GMT-2
+    os.setenv('TZ', 'GMT-2')
+    ffi.C.tzset()
+    test:ok(date.ctime(t) == 'Thu Jan  1 02:00:00 1970\n', ('%s: ctime with timezone'):format(str))
+    test:ok(date.strftime('%d/%m/%Y', t) == '01/01/1970', ('%s: strftime #1'):format(str))
+    test:ok(date.strftime('%A %d. %B %Y', t) == 'Thursday 01. January 1970', ('%s: strftime #2'):format(str))
+end)
+
+test:test("Parse iso date - valid strings", function(test)
+    test:plan(32)
+    local good = {
+        {2012, 12, 24, "20121224",                   8 },
+        {2012, 12, 24, "20121224  Foo bar",          8 },
+        {2012, 12, 24, "2012-12-24",                10 },
+        {2012, 12, 24, "2012-12-24 23:59:59",       10 },
+        {2012, 12, 24, "2012-12-24T00:00:00+00:00", 10 },
+        {2012, 12, 24, "2012359",                    7 },
+        {2012, 12, 24, "2012359T235959+0130",        7 },
+        {2012, 12, 24, "2012-359",                   8 },
+        {2012, 12, 24, "2012W521",                   8 },
+        {2012, 12, 24, "2012-W52-1",                10 },
+        {2012, 12, 24, "2012Q485",                   8 },
+        {2012, 12, 24, "2012-Q4-85",                10 },
+        {   1,  1,  1, "0001-Q1-01",                10 },
+        {   1,  1,  1, "0001-W01-1",                10 },
+        {   1,  1,  1, "0001-01-01",                10 },
+        {   1,  1,  1, "0001-001",                   8 },
+    }
+
+    for _, value in ipairs(good) do
+        local year, month, day, str, date_part_len
+        year, month, day, str, date_part_len = unpack(value)
+        local expected_date = date{year = year, month = month, day = day}
+        local date_part, len
+        date_part, len = date.parse_date(str)
+        test:ok(len == date_part_len, ('%s: length check %d'):format(str, len))
+        test:ok(expected_date == date_part, ('%s: expected date'):format(str))
+    end
+end)
+
+test:test("Parse iso date - invalid strings", function(test)
+    test:plan(62)
+    local bad = {
+        "20121232"   , -- Invalid day of month
+        "2012-12-310", -- Invalid day of month
+        "2012-13-24" , -- Invalid month
+        "2012367"    , -- Invalid day of year
+        "2012-000"   , -- Invalid day of year
+        "2012W533"   , -- Invalid week of year
+        "2012-W52-8" , -- Invalid day of week
+        "2012Q495"   , -- Invalid day of quarter
+        "2012-Q5-85" , -- Invalid quarter
+        "20123670"   , -- Trailing digit
+        "201212320"  , -- Trailing digit
+        "2012-12"    , -- Reduced accuracy
+        "2012-Q4"    , -- Reduced accuracy
+        "2012-Q42"   , -- Invalid
+        "2012-Q1-1"  , -- Invalid day of quarter
+        "2012Q--420" , -- Invalid
+        "2012-Q-420" , -- Invalid
+        "2012Q11"    , -- Incomplete
+        "2012Q1234"  , -- Trailing digit
+        "2012W12"    , -- Incomplete
+        "2012W1234"  , -- Trailing digit
+        "2012W-123"  , -- Invalid
+        "2012-W12"   , -- Incomplete
+        "2012-W12-12", -- Trailing digit
+        "2012U1234"  , -- Invalid
+        "2012-1234"  , -- Invalid
+        "2012-X1234" , -- Invalid
+        "0000-Q1-01" , -- Year less than 0001
+        "0000-W01-1" , -- Year less than 0001
+        "0000-01-01" , -- Year less than 0001
+        "0000-001"   , -- Year less than 0001
+    }
+
+    for _, str in ipairs(bad) do
+        local date_part, len
+        date_part, len = date.parse_date(str)
+        test:ok(len == 0, ('%s: length check %d'):format(str, len))
+        test:ok(date_part == nil, ('%s: empty date check %s'):format(str, date_part))
+    end
+end)
+
+test:test("Parse tiny date into seconds and other parts", function(test)
+    test:plan(9)
+    local str = '19700101 00:00:30.528'
+    local tiny = date(str)
+    test:ok(tiny.secs == 30, ("secs of '%s'"):format(str))
+    test:ok(tiny.nsec == 528000000, ("nsec of '%s'"):format(str))
+    test:ok(tiny.nanoseconds == 30528000000, "nanoseconds")
+    test:ok(tiny.microseconds == 30528000, "microseconds")
+    test:ok(tiny.milliseconds == 30528, "milliseconds")
+    test:ok(tiny.seconds == 30.528, "seconds")
+    test:ok(tiny.timestamp == 30.528, "timestamp")
+    test:ok(tiny.minutes == 0.5088, "minutes")
+    test:ok(tiny.hours == 0.00848, "hours")
+end)
+
+os.exit(test:check() and 0 or 1)
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 4/9] lua, datetime: display datetime
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (2 preceding siblings ...)
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 3/9] lua, datetime: datetime tests Timur Safin via Tarantool-patches
@ 2021-08-02  0:41 ` Timur Safin via Tarantool-patches
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
                   ` (7 subsequent siblings)
  11 siblings, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:41 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* introduced output routine for converting datetime
  to their default output format.

* use this routine for tostring() in datetime.lua

Part of #5941
---
 src/exports.h               |  1 +
 src/lib/core/CMakeLists.txt |  1 +
 src/lib/core/datetime.c     | 85 +++++++++++++++++++++++++++++++++++++
 src/lib/core/datetime.h     |  4 ++
 src/lua/datetime.lua        | 40 +++++++++++++++++
 5 files changed, 131 insertions(+)
 create mode 100755 src/lib/core/datetime.c

diff --git a/src/exports.h b/src/exports.h
index 2ec314369..ce8f74065 100644
--- a/src/exports.h
+++ b/src/exports.h
@@ -217,6 +217,7 @@ EXPORT(curl_url_set)
 EXPORT(curl_version)
 EXPORT(curl_version_info)
 #endif /* EXPORT_LIBCURL_SYMBOLS */
+EXPORT(datetime_to_string)
 EXPORT(decimal_unpack)
 EXPORT(dt_dow)
 EXPORT(dt_from_rdn)
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index 2cd4d0b4f..8bc776b82 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -30,6 +30,7 @@ set(core_sources
     decimal.c
     mp_decimal.c
     cord_buf.c
+    datetime.c
 )
 
 if (TARGET_OS_NETBSD)
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
new file mode 100755
index 000000000..65f813a70
--- /dev/null
+++ b/src/lib/core/datetime.c
@@ -0,0 +1,85 @@
+/*
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ *    copyright notice, this list of conditions and the
+ *    following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ *    copyright notice, this list of conditions and the following
+ *    disclaimer in the documentation and/or other materials
+ *    provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include <string.h>
+
+#include "trivia/util.h"
+#include "datetime.h"
+
+int
+datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
+{
+	char * src = buf;
+	int offset = date->offset;
+	int64_t secs = date->secs + offset * 60;
+	dt_t dt = dt_from_rdn((secs / SECS_PER_DAY) + 719163);
+
+	int year, month, day, sec, ns, sign;
+	dt_to_ymd(dt, &year, &month, &day);
+
+	int hour = (secs / 3600) % 24,
+	    minute = (secs / 60) % 60;
+	;
+	sec = secs % 60;
+	ns = date->nsec;
+	uint32_t sz;
+	sz = snprintf(buf, len, "%04d-%02d-%02dT%02d:%02d",
+		      year, month, day, hour, minute);
+	buf += sz; len -= sz;
+	if (sec || ns) {
+		sz = snprintf(buf, len, ":%02d", sec);
+		buf += sz; len -= sz;
+		if (ns) {
+			if ((ns % 1000000) == 0)
+				sz = snprintf(buf, len, ".%03d", ns / 1000000);
+			else if ((ns % 1000) == 0)
+				sz = snprintf(buf, len, ".%06d", ns / 1000);
+			else
+				sz = snprintf(buf, len, ".%09d", ns);
+			buf += sz; len -= sz;
+		}
+	}
+	if (offset == 0) {
+		strncpy(buf, "Z", len);
+		buf++;
+		len--;
+	}
+	else {
+		if (offset < 0)
+			sign = '-', offset = -offset;
+		else
+			sign = '+';
+
+		sz = snprintf(buf, len, "%c%02d:%02d", sign, offset / 60, offset % 60);
+		buf += sz; len -= sz;
+	}
+	return (buf - src);
+}
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
index 84d6642ab..2c9530ad7 100644
--- a/src/lib/core/datetime.h
+++ b/src/lib/core/datetime.h
@@ -40,6 +40,10 @@ extern "C"
 {
 #endif /* defined(__cplusplus) */
 
+#ifndef SECS_PER_DAY
+#define SECS_PER_DAY	86400
+#endif
+
 /**
  * Full datetime structure representing moments
  * since Unix Epoch (1970-01-01).
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
index 870edc541..e5b89768f 100644
--- a/src/lua/datetime.lua
+++ b/src/lua/datetime.lua
@@ -66,6 +66,12 @@ ffi.cdef [[
     dt_t    dt_from_struct_tm  (const struct tm *tm);
     void    dt_to_struct_tm    (dt_t dt, struct tm *tm);
 
+    // datetime.c
+
+    int
+    datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
+
+
     // <asm-generic/posix_types.h>
     typedef long            __kernel_long_t;
     typedef unsigned long   __kernel_ulong_t;
@@ -467,6 +473,36 @@ local function datetime_new(o)
     return mk_timestamp(dt, secs, frac, offset)
 end
 
+local function datetime_tostring(o)
+    if ffi.typeof(o) == datetime_t then
+        local sz = 48
+        local buff = ffi.new('char[?]', sz)
+        local len = builtin.datetime_to_string(o, buff, sz)
+        assert(len < sz)
+        return ffi.string(buff)
+    elseif ffi.typeof(o) == interval_t then
+        local ts = o.timestamp
+        local sign = '+'
+
+        if ts < 0 then
+            ts = -ts
+            sign = '-'
+        end
+
+        if ts < 60 then
+            return ('%s%s secs'):format(sign, ts)
+        elseif ts < 60 * 60 then
+            return ('%+d minutes, %s seconds'):format(o.minutes, ts % 60)
+        elseif ts < 24 * 60 * 60 then
+            return ('%+d hours, %d minutes, %s seconds'):format(
+                    o.hours, o.minutes % 60, ts % 60)
+        else
+            return ('%+d days, %d hours, %d minutes, %s seconds'):format(
+                    o.days, o.hours % 24, o.minutes % 60, ts % 60)
+        end
+    end
+end
+
 local function date_first(lhs, rhs)
     if is_datetime(lhs) then
         return lhs, rhs
@@ -710,6 +746,7 @@ local function strftime(fmt, o)
 end
 
 local datetime_mt = {
+    __tostring = datetime_tostring,
     __serialize = datetime_serialize,
     __eq = datetime_eq,
     __lt = datetime_lt,
@@ -721,6 +758,7 @@ local datetime_mt = {
 }
 
 local interval_mt = {
+    __tostring = datetime_tostring,
     __serialize = interval_serialize,
     __eq = datetime_eq,
     __lt = datetime_lt,
@@ -743,6 +781,8 @@ return setmetatable(
         parse_time  = parse_time,
         parse_zone  = parse_zone,
 
+        tostring    = datetime_tostring,
+
         now         = local_now,
         strftime    = strftime,
         asctime     = asctime,
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (3 preceding siblings ...)
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 4/9] lua, datetime: display datetime Timur Safin via Tarantool-patches
@ 2021-08-02  0:41 ` Timur Safin via Tarantool-patches
  2021-08-03 13:38   ` Timur Safin via Tarantool-patches
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
                   ` (6 subsequent siblings)
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:41 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

Serialize datetime_t as newly introduced MP_EXT type.
It saves 1 required integer field and upto 2 optional
unsigned fields in very compact fashion.
- secs is required field;
- but nsec, offset are both optional;

* json, yaml serialization formats, lua output mode
  supported;
* exported symbols for datetime messagepack size calculations
  so they are available for usage on Lua side.

Part of #5941
Part of #5946
---
 src/box/field_def.c               |  34 +++---
 src/box/field_def.h               |   1 +
 src/box/lua/serialize_lua.c       |   7 +-
 src/box/msgpack.c                 |   7 +-
 src/box/tuple_compare.cc          |  20 ++++
 src/exports.h                     |   2 +
 src/lib/core/CMakeLists.txt       |   3 +-
 src/lib/core/datetime.c           | 166 ++++++++++++++++++++++++++++++
 src/lib/core/mp_extension_types.h |   1 +
 src/lib/mpstream/mpstream.c       |  11 ++
 src/lib/mpstream/mpstream.h       |   4 +
 src/lua/msgpack.c                 |  12 +++
 src/lua/msgpackffi.lua            |  18 ++++
 src/lua/serializer.c              |   4 +
 src/lua/serializer.h              |   2 +
 src/lua/utils.c                   |   1 -
 test/unit/datetime.c              |   1 -
 third_party/lua-cjson/lua_cjson.c |   8 ++
 third_party/lua-yaml/lyaml.cc     |   6 +-
 19 files changed, 287 insertions(+), 21 deletions(-)

diff --git a/src/box/field_def.c b/src/box/field_def.c
index 51acb8025..6964e3e9f 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -67,11 +67,12 @@ const uint32_t field_mp_type[] = {
 	/* [FIELD_TYPE_VARBINARY] =  */ 1U << MP_BIN,
 	/* [FIELD_TYPE_SCALAR]   =  */ (1U << MP_UINT) | (1U << MP_INT) |
 		(1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_STR) |
-		(1U << MP_BIN) | (1U << MP_BOOL),
+		(1U << MP_BIN) | (1U << MP_BOOL) | (1U << MP_DATETIME),
 	/* [FIELD_TYPE_DECIMAL]  =  */ 0, /* only MP_DECIMAL is supported */
 	/* [FIELD_TYPE_UUID]     =  */ 0, /* only MP_UUID is supported */
 	/* [FIELD_TYPE_ARRAY]    =  */ 1U << MP_ARRAY,
 	/* [FIELD_TYPE_MAP]      =  */ (1U << MP_MAP),
+	/* [FIELD_TYPE_DATETIME] =  */ 0, /* only MP_DATETIME is supported */
 };
 
 const uint32_t field_ext_type[] = {
@@ -88,6 +89,7 @@ const uint32_t field_ext_type[] = {
 	/* [FIELD_TYPE_UUID]      = */ 1U << MP_UUID,
 	/* [FIELD_TYPE_ARRAY]     = */ 0,
 	/* [FIELD_TYPE_MAP]       = */ 0,
+	/* [FIELD_TYPE_DATETIME]  = */ 1U << MP_DATETIME,
 };
 
 const char *field_type_strs[] = {
@@ -104,6 +106,7 @@ const char *field_type_strs[] = {
 	/* [FIELD_TYPE_UUID]     = */ "uuid",
 	/* [FIELD_TYPE_ARRAY]    = */ "array",
 	/* [FIELD_TYPE_MAP]      = */ "map",
+	/* [FIELD_TYPE_DATETIME] = */ "datetime",
 };
 
 const char *on_conflict_action_strs[] = {
@@ -128,20 +131,21 @@ field_type_by_name_wrapper(const char *str, uint32_t len)
  * values can be stored in the j type.
  */
 static const bool field_type_compatibility[] = {
-	   /*   ANY   UNSIGNED  STRING   NUMBER  DOUBLE  INTEGER  BOOLEAN VARBINARY SCALAR  DECIMAL   UUID    ARRAY    MAP  */
-/*   ANY    */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   false,
-/* UNSIGNED */ true,   true,    false,   true,    false,   true,    false,   false,  true,   false,  false,   false,   false,
-/*  STRING  */ true,   false,   true,    false,   false,   false,   false,   false,  true,   false,  false,   false,   false,
-/*  NUMBER  */ true,   false,   false,   true,    false,   false,   false,   false,  true,   false,  false,   false,   false,
-/*  DOUBLE  */ true,   false,   false,   true,    true,    false,   false,   false,  true,   false,  false,   false,   false,
-/*  INTEGER */ true,   false,   false,   true,    false,   true,    false,   false,  true,   false,  false,   false,   false,
-/*  BOOLEAN */ true,   false,   false,   false,   false,   false,   true,    false,  true,   false,  false,   false,   false,
-/* VARBINARY*/ true,   false,   false,   false,   false,   false,   false,   true,   true,   false,  false,   false,   false,
-/*  SCALAR  */ true,   false,   false,   false,   false,   false,   false,   false,  true,   false,  false,   false,   false,
-/*  DECIMAL */ true,   false,   false,   true,    false,   false,   false,   false,  true,   true,   false,   false,   false,
-/*   UUID   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  true,    false,   false,
-/*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   true,    false,
-/*    MAP   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   true,
+	   /*   ANY   UNSIGNED  STRING   NUMBER  DOUBLE  INTEGER  BOOLEAN VARBINARY SCALAR  DECIMAL   UUID    ARRAY    MAP     DATETIME */
+/*   ANY    */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   false,   false,
+/* UNSIGNED */ true,   true,    false,   true,    false,   true,    false,   false,  true,   false,  false,   false,   false,   false,
+/*  STRING  */ true,   false,   true,    false,   false,   false,   false,   false,  true,   false,  false,   false,   false,   false,
+/*  NUMBER  */ true,   false,   false,   true,    false,   false,   false,   false,  true,   false,  false,   false,   false,   false,
+/*  DOUBLE  */ true,   false,   false,   true,    true,    false,   false,   false,  true,   false,  false,   false,   false,   false,
+/*  INTEGER */ true,   false,   false,   true,    false,   true,    false,   false,  true,   false,  false,   false,   false,   false,
+/*  BOOLEAN */ true,   false,   false,   false,   false,   false,   true,    false,  true,   false,  false,   false,   false,   false,
+/* VARBINARY*/ true,   false,   false,   false,   false,   false,   false,   true,   true,   false,  false,   false,   false,   false,
+/*  SCALAR  */ true,   false,   false,   false,   false,   false,   false,   false,  true,   false,  false,   false,   false,   false,
+/*  DECIMAL */ true,   false,   false,   true,    false,   false,   false,   false,  true,   true,   false,   false,   false,   false,
+/*   UUID   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  true,    false,   false,   false,
+/*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   true,    false,   false,
+/*    MAP   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   true,    false,
+/* DATETIME */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   false,   true,
 };
 
 bool
diff --git a/src/box/field_def.h b/src/box/field_def.h
index c5cfe5e86..120b2a93d 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -63,6 +63,7 @@ enum field_type {
 	FIELD_TYPE_UUID,
 	FIELD_TYPE_ARRAY,
 	FIELD_TYPE_MAP,
+	FIELD_TYPE_DATETIME,
 	field_type_MAX
 };
 
diff --git a/src/box/lua/serialize_lua.c b/src/box/lua/serialize_lua.c
index 7144305cf..5cfa6f5e0 100644
--- a/src/box/lua/serialize_lua.c
+++ b/src/box/lua/serialize_lua.c
@@ -768,7 +768,7 @@ static int
 dump_node(struct lua_dumper *d, struct node *nd, int indent)
 {
 	struct luaL_field *field = &nd->field;
-	char buf[FPCONV_G_FMT_BUFSIZE];
+	char buf[FPCONV_G_FMT_BUFSIZE + 8];
 	int ltype = lua_type(d->L, -1);
 	const char *str = NULL;
 	size_t len = 0;
@@ -861,6 +861,11 @@ dump_node(struct lua_dumper *d, struct node *nd, int indent)
 			str = tt_uuid_str(field->uuidval);
 			len = UUID_STR_LEN;
 			break;
+		case MP_DATETIME:
+			nd->mask |= NODE_QUOTE;
+			str = buf;
+			len = datetime_to_string(field->dateval, buf, sizeof buf);
+			break;
 		default:
 			d->err = EINVAL;
 			snprintf(d->err_msg, sizeof(d->err_msg),
diff --git a/src/box/msgpack.c b/src/box/msgpack.c
index 1723dea4c..e53af548c 100644
--- a/src/box/msgpack.c
+++ b/src/box/msgpack.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2020, Tarantool AUTHORS, please see AUTHORS file.
+ * Copyright 2020-2021, Tarantool AUTHORS, please see AUTHORS file.
  *
  * Redistribution and use in source and binary forms, with or
  * without modification, are permitted provided that the following
@@ -35,6 +35,7 @@
 #include "mp_decimal.h"
 #include "uuid/mp_uuid.h"
 #include "mp_error.h"
+#include "datetime.h"
 
 static int
 msgpack_fprint_ext(FILE *file, const char **data, int depth)
@@ -47,6 +48,8 @@ msgpack_fprint_ext(FILE *file, const char **data, int depth)
 		return mp_fprint_decimal(file, data, len);
 	case MP_UUID:
 		return mp_fprint_uuid(file, data, len);
+	case MP_DATETIME:
+		return mp_fprint_datetime(file, data, len);
 	case MP_ERROR:
 		return mp_fprint_error(file, data, depth);
 	default:
@@ -65,6 +68,8 @@ msgpack_snprint_ext(char *buf, int size, const char **data, int depth)
 		return mp_snprint_decimal(buf, size, data, len);
 	case MP_UUID:
 		return mp_snprint_uuid(buf, size, data, len);
+	case MP_DATETIME:
+		return mp_snprint_datetime(buf, size, data, len);
 	case MP_ERROR:
 		return mp_snprint_error(buf, size, data, depth);
 	default:
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index 98938fb39..48c833643 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -36,6 +36,7 @@
 #include "lib/core/decimal.h"
 #include "lib/core/mp_decimal.h"
 #include "uuid/mp_uuid.h"
+#include "core/datetime.h"
 #include "lib/core/mp_extension_types.h"
 
 /* {{{ tuple_compare */
@@ -76,6 +77,7 @@ enum mp_class {
 	MP_CLASS_STR,
 	MP_CLASS_BIN,
 	MP_CLASS_UUID,
+	MP_CLASS_DATETIME,
 	MP_CLASS_ARRAY,
 	MP_CLASS_MAP,
 	mp_class_max,
@@ -99,6 +101,8 @@ static enum mp_class mp_ext_classes[] = {
 	/* .MP_UNKNOWN_EXTENSION = */ mp_class_max, /* unsupported */
 	/* .MP_DECIMAL		 = */ MP_CLASS_NUMBER,
 	/* .MP_UUID		 = */ MP_CLASS_UUID,
+	/* .MP_ERROR		 = */ mp_class_max,
+	/* .MP_DATETIME		 = */ MP_CLASS_DATETIME,
 };
 
 static enum mp_class
@@ -390,6 +394,19 @@ mp_compare_uuid(const char *field_a, const char *field_b)
 	return memcmp(field_a + 2, field_b + 2, UUID_PACKED_LEN);
 }
 
+static int
+mp_compare_datetime(const char *lhs, const char *rhs)
+{
+	datetime_t lhs_dt, rhs_dt;
+	datetime_t *ret;
+	ret = mp_decode_datetime(&lhs, &lhs_dt);
+	assert(ret != NULL);
+	ret = mp_decode_datetime(&rhs, &rhs_dt);
+	assert(ret != NULL);
+	(void)ret;
+	return datetime_compare(&lhs_dt, &rhs_dt);
+}
+
 typedef int (*mp_compare_f)(const char *, const char *);
 static mp_compare_f mp_class_comparators[] = {
 	/* .MP_CLASS_NIL    = */ NULL,
@@ -398,6 +415,7 @@ static mp_compare_f mp_class_comparators[] = {
 	/* .MP_CLASS_STR    = */ mp_compare_str,
 	/* .MP_CLASS_BIN    = */ mp_compare_bin,
 	/* .MP_CLASS_UUID   = */ mp_compare_uuid,
+	/* .MP_CLASS_DATETIME=*/ mp_compare_datetime,
 	/* .MP_CLASS_ARRAY  = */ NULL,
 	/* .MP_CLASS_MAP    = */ NULL,
 };
@@ -478,6 +496,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
 		return mp_compare_decimal(field_a, field_b);
 	case FIELD_TYPE_UUID:
 		return mp_compare_uuid(field_a, field_b);
+	case FIELD_TYPE_DATETIME:
+		return mp_compare_datetime(field_a, field_b);
 	default:
 		unreachable();
 		return 0;
diff --git a/src/exports.h b/src/exports.h
index ce8f74065..7ac2cd012 100644
--- a/src/exports.h
+++ b/src/exports.h
@@ -459,6 +459,7 @@ EXPORT(mp_encode_decimal)
 EXPORT(mp_encode_double)
 EXPORT(mp_encode_float)
 EXPORT(mp_encode_uuid)
+EXPORT(mp_encode_datetime)
 EXPORT(mp_ext_hint)
 EXPORT(mp_format)
 EXPORT(mp_fprint)
@@ -468,6 +469,7 @@ EXPORT(mp_fprint_recursion)
 EXPORT(mp_parser_hint)
 EXPORT(mp_sizeof_decimal)
 EXPORT(mp_sizeof_uuid)
+EXPORT(mp_sizeof_datetime)
 EXPORT(mp_snprint)
 EXPORT(mp_snprint_ext)
 EXPORT(mp_snprint_ext_default)
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index 8bc776b82..c3e68c36f 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -44,7 +44,8 @@ add_library(core STATIC ${core_sources})
 
 target_link_libraries(core salad small uri decNumber bit ${LIBEV_LIBRARIES}
                       ${LIBEIO_LIBRARIES} ${LIBCORO_LIBRARIES}
-                      ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES})
+                      ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES}
+                      ${LIBCDT_LIBRARIES})
 
 if (ENABLE_BACKTRACE AND NOT TARGET_OS_DARWIN)
     target_link_libraries(core gcc_s ${UNWIND_LIBRARIES})
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
index 65f813a70..96e554bd2 100755
--- a/src/lib/core/datetime.c
+++ b/src/lib/core/datetime.c
@@ -33,6 +33,137 @@
 
 #include "trivia/util.h"
 #include "datetime.h"
+#include "msgpuck.h"
+#include "mp_extension_types.h"
+
+/*
+  Datetime MessagePack serialization schema is MP_EXT (0xC7 for 1 byte length)
+  extension, which creates container of 1 to 3 integers.
+
+  +----+---+-----------+====~~~~~~~====+-----~~~~~~~~-------+....~~~~~~~....+
+  |0xC7| 4 |len (uint8)| seconds (int) | nanoseconds (uint) | offset (uint) |
+  +----+---+-----------+====~~~~~~~====+-----~~~~~~~~-------+....~~~~~~~....+
+
+  MessagePack extension MP_EXT (0xC7), after 1-byte length, contains:
+
+  - signed integer seconds part (required). Depending on the value of
+    seconds it may be from 1 to 8 bytes positive or negative integer number;
+
+  - [optional] fraction time in nanoseconds as unsigned integer.
+    If this value is 0 then it's not saved (unless there is offset field,
+    as below);
+
+  - [optional] timzeone offset in minutes as unsigned integer.
+    If this field is 0 then it's not saved.
+ */
+
+static inline uint32_t
+mp_sizeof_Xint(int64_t n)
+{
+	return n < 0 ? mp_sizeof_int(n) : mp_sizeof_uint(n);
+}
+
+static inline char *
+mp_encode_Xint(char *data, int64_t v)
+{
+	return v < 0 ? mp_encode_int(data, v) : mp_encode_uint(data, v);
+}
+
+static inline int64_t
+mp_decode_Xint(const char **data)
+{
+	switch (mp_typeof(**data)) {
+	case MP_UINT:
+		return (int64_t)mp_decode_uint(data);
+	case MP_INT:
+		return mp_decode_int(data);
+	default:
+		mp_unreachable();
+	}
+	return 0;
+}
+
+static inline uint32_t
+mp_sizeof_datetime_raw(const struct datetime_t *date)
+{
+	uint32_t sz = mp_sizeof_Xint(date->secs);
+
+	// even if nanosecs == 0 we need to output anything
+	// if we have non-null tz offset
+	if (date->nsec != 0 || date->offset != 0)
+		sz += mp_sizeof_Xint(date->nsec);
+	if (date->offset)
+		sz += mp_sizeof_Xint(date->offset);
+	return sz;
+}
+
+uint32_t
+mp_sizeof_datetime(const struct datetime_t *date)
+{
+	return mp_sizeof_ext(mp_sizeof_datetime_raw(date));
+}
+
+struct datetime_t *
+datetime_unpack(const char **data, uint32_t len, struct datetime_t *date)
+{
+	const char * svp = *data;
+
+	memset(date, 0, sizeof(*date));
+
+	date->secs = mp_decode_Xint(data);
+
+	len -= *data - svp;
+	if (len <= 0)
+		return date;
+
+	svp = *data;
+	date->nsec = mp_decode_Xint(data);
+	len -= *data - svp;
+
+	if (len <= 0)
+		return date;
+
+	date->offset = mp_decode_Xint(data);
+
+	return date;
+}
+
+struct datetime_t *
+mp_decode_datetime(const char **data, struct datetime_t *date)
+{
+	if (mp_typeof(**data) != MP_EXT)
+		return NULL;
+
+	int8_t type;
+	uint32_t len = mp_decode_extl(data, &type);
+
+	if (type != MP_DATETIME || len == 0) {
+		return NULL;
+	}
+	return datetime_unpack(data, len, date);
+}
+
+char *
+datetime_pack(char *data, const struct datetime_t *date)
+{
+	data = mp_encode_Xint(data, date->secs);
+	if (date->nsec != 0 || date->offset != 0)
+		data = mp_encode_Xint(data, date->nsec);
+	if (date->offset)
+		data = mp_encode_Xint(data, date->offset);
+
+	return data;
+}
+
+char *
+mp_encode_datetime(char *data, const struct datetime_t *date)
+{
+	uint32_t len = mp_sizeof_datetime_raw(date);
+
+	data = mp_encode_extl(data, MP_DATETIME, len);
+
+	return datetime_pack(data, date);
+}
 
 int
 datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
@@ -83,3 +214,38 @@ datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
 	}
 	return (buf - src);
 }
+
+int
+mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
+{
+	struct datetime_t date = {0};
+
+	if (datetime_unpack(data, len, &date) == NULL)
+		return -1;
+
+	return datetime_to_string(&date, buf, size);
+}
+
+int
+mp_fprint_datetime(FILE *file, const char **data, uint32_t len)
+{
+	struct  datetime_t date;
+
+	if (datetime_unpack(data, len, &date) == NULL)
+		return -1;
+
+	char buf[48];
+	datetime_to_string(&date, buf, sizeof buf);
+
+	return fprintf(file, "%s", buf);
+}
+
+int datetime_compare(const struct datetime_t *lhs, const struct datetime_t *rhs)
+{
+	int result = COMPARE_RESULT(lhs->secs, rhs->secs);
+	if (result != 0)
+		return result;
+
+	return COMPARE_RESULT(lhs->nsec, rhs->nsec);
+}
+
diff --git a/src/lib/core/mp_extension_types.h b/src/lib/core/mp_extension_types.h
index e3ff9f5d0..3b7eaee7c 100644
--- a/src/lib/core/mp_extension_types.h
+++ b/src/lib/core/mp_extension_types.h
@@ -44,6 +44,7 @@ enum mp_extension_type {
     MP_DECIMAL = 1,
     MP_UUID = 2,
     MP_ERROR = 3,
+    MP_DATETIME = 4,
     mp_extension_type_MAX,
 };
 
diff --git a/src/lib/mpstream/mpstream.c b/src/lib/mpstream/mpstream.c
index 70ca29889..1077e3b19 100644
--- a/src/lib/mpstream/mpstream.c
+++ b/src/lib/mpstream/mpstream.c
@@ -35,6 +35,7 @@
 #include "msgpuck.h"
 #include "mp_decimal.h"
 #include "uuid/mp_uuid.h"
+#include "core/datetime.h"
 
 void
 mpstream_reserve_slow(struct mpstream *stream, size_t size)
@@ -208,6 +209,16 @@ mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid)
 	mpstream_advance(stream, pos - data);
 }
 
+void
+mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *val)
+{
+	char *data = mpstream_reserve(stream, mp_sizeof_datetime(val));
+	if (data == NULL)
+		return;
+	char *pos = mp_encode_datetime(data, val);
+	mpstream_advance(stream, pos - data);
+}
+
 void
 mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n)
 {
diff --git a/src/lib/mpstream/mpstream.h b/src/lib/mpstream/mpstream.h
index a60add143..540e9a666 100644
--- a/src/lib/mpstream/mpstream.h
+++ b/src/lib/mpstream/mpstream.h
@@ -39,6 +39,7 @@ extern "C" {
 #endif /* defined(__cplusplus) */
 
 struct tt_uuid;
+struct datetime_t;
 
 /**
 * Ask the allocator to reserve at least size bytes. It can reserve
@@ -145,6 +146,9 @@ mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val);
 void
 mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid);
 
+void
+mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *dt);
+
 /** Copies n bytes from memory area src to stream. */
 void
 mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n);
diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
index b6ecf2b1e..0a4ba8129 100644
--- a/src/lua/msgpack.c
+++ b/src/lua/msgpack.c
@@ -46,6 +46,7 @@
 #include "lib/core/decimal.h" /* decimal_unpack() */
 #include "lib/uuid/mp_uuid.h" /* mp_decode_uuid() */
 #include "lib/core/mp_extension_types.h"
+#include "datetime.h"
 
 #include "cord_buf.h"
 #include <fiber.h>
@@ -200,6 +201,9 @@ restart: /* used by MP_EXT of unidentified subtype */
 			break;
 		case MP_ERROR:
 			return luamp_encode_extension(L, top, stream);
+		case MP_DATETIME:
+			mpstream_encode_datetime(stream, field->dateval);
+			break;
 		default:
 			/* Run trigger if type can't be encoded */
 			type = luamp_encode_extension(L, top, stream);
@@ -333,6 +337,14 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
 				goto ext_decode_err;
 			return;
 		}
+		case MP_DATETIME:
+		{
+			struct datetime_t * date = luaL_pushdatetime(L);
+			date = datetime_unpack(data, len, date);
+			if (date == NULL)
+				goto ext_decode_err;
+			return;
+		}
 		default:
 			/* reset data to the extension header */
 			*data = svp;
diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua
index 1d54f11b8..c47d77acb 100644
--- a/src/lua/msgpackffi.lua
+++ b/src/lua/msgpackffi.lua
@@ -26,6 +26,10 @@ char *
 mp_encode_uuid(char *data, const struct tt_uuid *uuid);
 uint32_t
 mp_sizeof_uuid();
+uint32_t
+mp_sizeof_datetime(const struct t_datetime_tz *date);
+char *
+mp_encode_datetime(char *data, const struct t_datetime_tz *date);
 float
 mp_decode_float(const char **data);
 double
@@ -36,6 +40,8 @@ decimal_t *
 decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
 struct tt_uuid *
 uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid);
+struct datetime_t *
+datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
 ]])
 
 local strict_alignment = (jit.arch == 'arm')
@@ -142,6 +148,11 @@ local function encode_uuid(buf, uuid)
     builtin.mp_encode_uuid(p, uuid)
 end
 
+local function encode_datetime(buf, date)
+    local p = buf:alloc(builtin.mp_sizeof_datetime(date))
+    builtin.mp_encode_datetime(p, date)
+end
+
 local function encode_int(buf, num)
     if num >= 0 then
         if num <= 0x7f then
@@ -320,6 +331,7 @@ on_encode(ffi.typeof('float'), encode_float)
 on_encode(ffi.typeof('double'), encode_double)
 on_encode(ffi.typeof('decimal_t'), encode_decimal)
 on_encode(ffi.typeof('struct tt_uuid'), encode_uuid)
+on_encode(ffi.typeof('struct datetime_t'), encode_datetime)
 
 --------------------------------------------------------------------------------
 -- Decoder
@@ -513,6 +525,12 @@ local ext_decoder = {
         builtin.uuid_unpack(data, len, uuid)
         return uuid
     end,
+    -- MP_DATETIME
+    [4] = function(data, len)
+        local dt = ffi.new("struct datetime_t")
+        builtin.datetime_unpack(data, len, dt)
+        return dt
+    end,
 }
 
 local function decode_ext(data)
diff --git a/src/lua/serializer.c b/src/lua/serializer.c
index 8db6746a3..c27e62c62 100644
--- a/src/lua/serializer.c
+++ b/src/lua/serializer.c
@@ -41,6 +41,7 @@
 #include "lib/core/mp_extension_types.h"
 #include "lua/error.h"
 
+#include "datetime.h"
 #include "trivia/util.h"
 #include "diag.h"
 #include "serializer_opts.h"
@@ -544,6 +545,9 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg,
 				   opts != NULL &&
 				   opts->error_marshaling_enabled) {
 				field->ext_type = MP_ERROR;
+			} else if (cd->ctypeid == CTID_DATETIME) {
+				field->ext_type = MP_DATETIME;
+				field->dateval = (struct datetime_t*) cdata;
 			} else {
 				field->ext_type = MP_UNKNOWN_EXTENSION;
 			}
diff --git a/src/lua/serializer.h b/src/lua/serializer.h
index 0a0501a74..52e51d279 100644
--- a/src/lua/serializer.h
+++ b/src/lua/serializer.h
@@ -52,6 +52,7 @@ extern "C" {
 #include <lauxlib.h>
 
 #include "trigger.h"
+#include "lib/core/datetime.h"
 #include "lib/core/decimal.h" /* decimal_t */
 #include "lib/core/mp_extension_types.h"
 #include "lua/error.h"
@@ -223,6 +224,7 @@ struct luaL_field {
 		uint32_t size;
 		decimal_t *decval;
 		struct tt_uuid *uuidval;
+		struct datetime_t *dateval;
 	};
 	enum mp_type type;
 	/* subtypes of MP_EXT */
diff --git a/src/lua/utils.c b/src/lua/utils.c
index 611044b6f..685cab47b 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -241,7 +241,6 @@ luaL_setcdatagc(struct lua_State *L, int idx)
 	lua_pop(L, 1);
 }
 
-
 /**
  * A helper to register a single type metatable.
  */
diff --git a/test/unit/datetime.c b/test/unit/datetime.c
index b6f568c03..226b6fadb 100644
--- a/test/unit/datetime.c
+++ b/test/unit/datetime.c
@@ -139,7 +139,6 @@ exit:
 // avoid introducing external datetime.h dependency
 // - just copy paste it for today
 #define SECS_PER_DAY      86400
-#define NANOS_PER_SEC     1000000000
 #define DT_EPOCH_1970_OFFSET 719163
 
 
diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c
index 5123b9a74..d5ddc4ea0 100644
--- a/third_party/lua-cjson/lua_cjson.c
+++ b/third_party/lua-cjson/lua_cjson.c
@@ -52,6 +52,7 @@
 #include "mp_extension_types.h" /* MP_DECIMAL, MP_UUID */
 #include "tt_static.h"
 #include "uuid/tt_uuid.h" /* tt_uuid_to_string(), UUID_STR_LEN */
+#include "core/datetime.h"
 #include "cord_buf.h"
 
 typedef enum {
@@ -426,6 +427,13 @@ static void json_append_data(lua_State *l, struct luaL_serializer *cfg,
         case MP_UUID:
             return json_append_string(cfg, json, tt_uuid_str(field.uuidval),
                                       UUID_STR_LEN);
+
+        case MP_DATETIME:
+        {
+            char buf[128];
+            size_t sz = datetime_to_string(field.dateval, buf, sizeof buf);
+            return json_append_string(cfg, json, buf, sz);
+        }
         default:
             assert(false);
         }
diff --git a/third_party/lua-yaml/lyaml.cc b/third_party/lua-yaml/lyaml.cc
index 5469e9f4f..39127fccf 100644
--- a/third_party/lua-yaml/lyaml.cc
+++ b/third_party/lua-yaml/lyaml.cc
@@ -617,7 +617,7 @@ static int dump_node(struct lua_yaml_dumper *dumper)
    yaml_event_t ev;
    yaml_scalar_style_t style = YAML_PLAIN_SCALAR_STYLE;
    int is_binary = 0;
-   char buf[FPCONV_G_FMT_BUFSIZE];
+   char buf[FPCONV_G_FMT_BUFSIZE + 8];
    struct luaL_field field;
    bool unused;
    (void) unused;
@@ -707,6 +707,10 @@ static int dump_node(struct lua_yaml_dumper *dumper)
          str = tt_uuid_str(field.uuidval);
          len = UUID_STR_LEN;
          break;
+      case MP_DATETIME:
+         len = datetime_to_string(field.dateval, buf, sizeof buf);
+         str = buf;
+         break;
       default:
          assert(0); /* checked by luaL_checkfield() */
       }
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (4 preceding siblings ...)
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
@ 2021-08-02  0:41 ` Timur Safin via Tarantool-patches
  2021-08-03 12:02   ` Serge Petrenko via Tarantool-patches
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 7/9] lua, datetime: time intervals support Timur Safin via Tarantool-patches
                   ` (5 subsequent siblings)
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:41 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* storage hints implemented for datetime_t values;
* proper comparison for indices of datetime type.

Part of #5941
Part of #5946
---
 src/box/field_def.c           | 18 ++++++++
 src/box/field_def.h           |  3 ++
 src/box/memtx_space.c         |  3 +-
 src/box/tuple_compare.cc      | 56 +++++++++++++++++++++++++
 src/box/vinyl.c               |  3 +-
 test/engine/datetime.result   | 77 +++++++++++++++++++++++++++++++++++
 test/engine/datetime.test.lua | 35 ++++++++++++++++
 7 files changed, 191 insertions(+), 4 deletions(-)
 create mode 100644 test/engine/datetime.result
 create mode 100644 test/engine/datetime.test.lua

diff --git a/src/box/field_def.c b/src/box/field_def.c
index 6964e3e9f..aaf5f9cff 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -193,3 +193,21 @@ field_type_by_name(const char *name, size_t len)
 		return FIELD_TYPE_ANY;
 	return field_type_MAX;
 }
+
+const bool field_type_index_allowed[] =
+    {
+	/* [FIELD_TYPE_ANY]      = */ false,
+	/* [FIELD_TYPE_UNSIGNED] = */ true,
+	/* [FIELD_TYPE_STRING]   = */ true,
+	/* [FIELD_TYPE_NUMBER]   = */ true,
+	/* [FIELD_TYPE_DOUBLE]   = */ true,
+	/* [FIELD_TYPE_INTEGER]  = */ true,
+	/* [FIELD_TYPE_BOOLEAN]  = */ true,
+	/* [FIELD_TYPE_VARBINARY]= */ true,
+	/* [FIELD_TYPE_SCALAR]   = */ true,
+	/* [FIELD_TYPE_DECIMAL]  = */ true,
+	/* [FIELD_TYPE_UUID]     = */ true,
+	/* [FIELD_TYPE_ARRAY]    = */ false,
+	/* [FIELD_TYPE_MAP]      = */ false,
+	/* [FIELD_TYPE_DATETIME] = */ true,
+};
diff --git a/src/box/field_def.h b/src/box/field_def.h
index 120b2a93d..bd02418df 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[];
 extern const struct opt_def field_def_reg[];
 extern const struct field_def field_def_default;
 
+/** helper table for checking allowed indices for types */
+extern const bool field_type_index_allowed[];
+
 /**
  * @brief Field definition
  * Contains information about of one tuple field.
diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c
index b71318d24..1ab16122e 100644
--- a/src/box/memtx_space.c
+++ b/src/box/memtx_space.c
@@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
 	/* Check that there are no ANY, ARRAY, MAP parts */
 	for (uint32_t i = 0; i < key_def->part_count; i++) {
 		struct key_part *part = &key_def->parts[i];
-		if (part->type <= FIELD_TYPE_ANY ||
-		    part->type >= FIELD_TYPE_ARRAY) {
+		if (!field_type_index_allowed[part->type]) {
 			diag_set(ClientError, ER_MODIFY_INDEX,
 				 index_def->name, space_name(space),
 				 tt_sprintf("field type '%s' is not supported",
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index 48c833643..f733b9f01 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
 						   field_b, b_type);
 	case FIELD_TYPE_UUID:
 		return mp_compare_uuid(field_a, field_b);
+	case FIELD_TYPE_DATETIME:
+		return mp_compare_datetime(field_a, field_b);
 	default:
 		unreachable();
 		return 0;
@@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint,
 #define HINT_VALUE_DOUBLE_MAX	(exp2(HINT_VALUE_BITS - 1) - 1)
 #define HINT_VALUE_DOUBLE_MIN	(-exp2(HINT_VALUE_BITS - 1))
 
+/**
+ * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds
+ * into 60 bits of hint value. The idea is to represent wide enough
+ * years range, and leave the rest of bits occupied from nanoseconds part:
+ * - 36 bits is enough for time range of [208BC..4147]
+ * - for nanoseconds there is left 24 bits, which are MSB part of
+ *   32-bit value
+ */
+#define HINT_VALUE_SECS_BITS	36
+#define HINT_VALUE_NSEC_BITS	(HINT_VALUE_BITS - HINT_VALUE_SECS_BITS)
+#define HINT_VALUE_SECS_MAX	((1LL << HINT_VALUE_SECS_BITS) - 1)
+#define HINT_VALUE_SECS_MIN	(-(1LL << HINT_VALUE_SECS_BITS))
+#define HINT_VALUE_NSEC_SHIFT	(sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS)
+#define HINT_VALUE_NSEC_MAX	((1ULL << HINT_VALUE_NSEC_BITS) - 1)
+
 /*
  * HINT_CLASS_BITS should be big enough to store any mp_class value.
  * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE.
@@ -1630,6 +1647,24 @@ hint_uuid_raw(const char *data)
 	return hint_create(MP_CLASS_UUID, val);
 }
 
+static inline hint_t
+hint_datetime(struct datetime_t *date)
+{
+	/*
+	 * Use at most HINT_VALUE_SECS_BITS from datetime
+	 * seconds field as a hint value, and at MSB part
+	 * of HINT_VALUE_NSEC_BITS from nanoseconds.
+	 */
+	int64_t secs = date->secs;
+	int32_t nsec = date->nsec;
+	uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 :
+			secs - HINT_VALUE_SECS_MIN;
+	val &= HINT_VALUE_SECS_MAX;
+	val <<= HINT_VALUE_NSEC_BITS;
+	val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX;
+	return hint_create(MP_CLASS_DATETIME, val);
+}
+
 static inline uint64_t
 hint_str_raw(const char *s, uint32_t len)
 {
@@ -1761,6 +1796,17 @@ field_hint_uuid(const char *field)
 	return hint_uuid_raw(data);
 }
 
+static inline hint_t
+field_hint_datetime(const char *field)
+{
+	assert(mp_typeof(*field) == MP_EXT);
+	int8_t ext_type;
+	uint32_t len = mp_decode_extl(&field, &ext_type);
+	assert(ext_type == MP_DATETIME);
+	struct datetime_t date;
+	return hint_datetime(datetime_unpack(&field, len, &date));
+}
+
 static inline hint_t
 field_hint_string(const char *field, struct coll *coll)
 {
@@ -1812,6 +1858,11 @@ field_hint_scalar(const char *field, struct coll *coll)
 		}
 		case MP_UUID:
 			return hint_uuid_raw(field);
+		case MP_DATETIME:
+		{
+			struct datetime_t date;
+			return hint_datetime(datetime_unpack(&field, len, &date));
+		}
 		default:
 			unreachable();
 		}
@@ -1849,6 +1900,8 @@ field_hint(const char *field, struct coll *coll)
 		return field_hint_decimal(field);
 	case FIELD_TYPE_UUID:
 		return field_hint_uuid(field);
+	case FIELD_TYPE_DATETIME:
+		return field_hint_datetime(field);
 	default:
 		unreachable();
 	}
@@ -1963,6 +2016,9 @@ key_def_set_hint_func(struct key_def *def)
 	case FIELD_TYPE_UUID:
 		key_def_set_hint_func<FIELD_TYPE_UUID>(def);
 		break;
+	case FIELD_TYPE_DATETIME:
+		key_def_set_hint_func<FIELD_TYPE_DATETIME>(def);
+		break;
 	default:
 		/* Invalid key definition. */
 		def->key_hint = NULL;
diff --git a/src/box/vinyl.c b/src/box/vinyl.c
index c80b2d99b..360d1fa70 100644
--- a/src/box/vinyl.c
+++ b/src/box/vinyl.c
@@ -662,8 +662,7 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def)
 	/* Check that there are no ANY, ARRAY, MAP parts */
 	for (uint32_t i = 0; i < key_def->part_count; i++) {
 		struct key_part *part = &key_def->parts[i];
-		if (part->type <= FIELD_TYPE_ANY ||
-		    part->type >= FIELD_TYPE_ARRAY) {
+		if (!field_type_index_allowed[part->type]) {
 			diag_set(ClientError, ER_MODIFY_INDEX,
 				 index_def->name, space_name(space),
 				 tt_sprintf("field type '%s' is not supported",
diff --git a/test/engine/datetime.result b/test/engine/datetime.result
new file mode 100644
index 000000000..848a0aaec
--- /dev/null
+++ b/test/engine/datetime.result
@@ -0,0 +1,77 @@
+-- test-run result file version 2
+env = require('test_run')
+ | ---
+ | ...
+test_run = env.new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+
+date = require('datetime')
+ | ---
+ | ...
+
+_ = box.schema.space.create('T', {engine = engine})
+ | ---
+ | ...
+_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
+ | ---
+ | ...
+
+box.space.T:insert{date('1970-01-01')}\
+box.space.T:insert{date('1970-01-02')}\
+box.space.T:insert{date('1970-01-03')}\
+box.space.T:insert{date('2000-01-01')}
+ | ---
+ | ...
+
+o = box.space.T:select{}
+ | ---
+ | ...
+assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
+ | ---
+ | - true
+ | ...
+assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
+ | ---
+ | - true
+ | ...
+assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
+ | ---
+ | - true
+ | ...
+assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
+ | ---
+ | - true
+ | ...
+
+for i = 1,16 do\
+    box.space.T:insert{date.now()}\
+end
+ | ---
+ | ...
+
+a = box.space.T:select{}
+ | ---
+ | ...
+err = {}
+ | ---
+ | ...
+for i = 1, #a - 1 do\
+    if a[i][1] >= a[i+1][1] then\
+        table.insert(err, {a[i][1], a[i+1][1]})\
+        break\
+    end\
+end
+ | ---
+ | ...
+
+err
+ | ---
+ | - []
+ | ...
+box.space.T:drop()
+ | ---
+ | ...
diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua
new file mode 100644
index 000000000..3685e4d4b
--- /dev/null
+++ b/test/engine/datetime.test.lua
@@ -0,0 +1,35 @@
+env = require('test_run')
+test_run = env.new()
+engine = test_run:get_cfg('engine')
+
+date = require('datetime')
+
+_ = box.schema.space.create('T', {engine = engine})
+_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
+
+box.space.T:insert{date('1970-01-01')}\
+box.space.T:insert{date('1970-01-02')}\
+box.space.T:insert{date('1970-01-03')}\
+box.space.T:insert{date('2000-01-01')}
+
+o = box.space.T:select{}
+assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
+assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
+assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
+assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
+
+for i = 1,16 do\
+    box.space.T:insert{date.now()}\
+end
+
+a = box.space.T:select{}
+err = {}
+for i = 1, #a - 1 do\
+    if a[i][1] >= a[i+1][1] then\
+        table.insert(err, {a[i][1], a[i+1][1]})\
+        break\
+    end\
+end
+
+err
+box.space.T:drop()
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 7/9] lua, datetime: time intervals support
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (5 preceding siblings ...)
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
@ 2021-08-02  0:41 ` Timur Safin via Tarantool-patches
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 8/9] datetime: changelog for datetime module Timur Safin via Tarantool-patches
                   ` (4 subsequent siblings)
  11 siblings, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:41 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

* created few entry points (months(N), years(N), days(N), etc.)
  for easier datetime arithmetic;
* additions/subtractions of years/months use `dt_add_years()`
  and `dt_add_months()` from 3rd party c-dt library;
* also there are `:add{}` and `:sub{}` methods in datetime
  object to add or substract more complex intervals;
* introduced `is_datetime()` and `is_interval()` helpers for checking
  of validity of passed arguments;
* human-readable stringization implemented for interval objects.

Note, that additions/subtractions completed for all _reasonable_
combinations of values of date and interval types;

	Time + Interval	=> Time
	Interval + Time => Time
	Time - Time 	=> Interval
	Time - Interval => Time
	Interval + Interval => Interval
	Interval - Interval => Interval

Part of #5941
---
 src/exports.h                  |   5 +
 src/lua/datetime.lua           | 276 ++++++++++++++++++++++++++++++++-
 test/app-tap/datetime.test.lua | 163 ++++++++++++++++++-
 3 files changed, 436 insertions(+), 8 deletions(-)

diff --git a/src/exports.h b/src/exports.h
index 7ac2cd012..63efe0ec7 100644
--- a/src/exports.h
+++ b/src/exports.h
@@ -217,8 +217,13 @@ EXPORT(curl_url_set)
 EXPORT(curl_version)
 EXPORT(curl_version_info)
 #endif /* EXPORT_LIBCURL_SYMBOLS */
+EXPORT(datetime_pack)
 EXPORT(datetime_to_string)
+EXPORT(datetime_unpack)
 EXPORT(decimal_unpack)
+EXPORT(dt_add_months)
+EXPORT(dt_add_quarters)
+EXPORT(dt_add_years)
 EXPORT(dt_dow)
 EXPORT(dt_from_rdn)
 EXPORT(dt_from_struct_tm)
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
index e5b89768f..dc88a9d9d 100644
--- a/src/lua/datetime.lua
+++ b/src/lua/datetime.lua
@@ -50,6 +50,17 @@ ffi.cdef [[
     int      dt_rdn          (dt_t dt);
     dt_dow_t dt_dow          (dt_t dt);
 
+    // dt_arithmetic.h
+    typedef enum {
+        DT_EXCESS,
+        DT_LIMIT,
+        DT_SNAP
+    } dt_adjust_t;
+
+    dt_t    dt_add_years        (dt_t dt, int delta, dt_adjust_t adjust);
+    dt_t    dt_add_quarters     (dt_t dt, int delta, dt_adjust_t adjust);
+    dt_t    dt_add_months       (dt_t dt, int delta, dt_adjust_t adjust);
+
     // dt_parse_iso.h
     size_t dt_parse_iso_date          (const char *str, size_t len, dt_t *dt);
 
@@ -171,9 +182,23 @@ local DT_EPOCH_1970_OFFSET = 719163LL
 
 local datetime_t = ffi.typeof('struct datetime_t')
 local interval_t = ffi.typeof('struct datetime_interval_t')
+ffi.cdef [[
+    struct t_interval_months {
+        int m;
+    };
+
+    struct t_interval_years {
+        int y;
+    };
+]]
+local interval_months_t = ffi.typeof('struct t_interval_months')
+local interval_years_t = ffi.typeof('struct t_interval_years')
 
 local function is_interval(o)
-    return type(o) == 'cdata' and ffi.istype(interval_t, o)
+    return type(o) == 'cdata' and
+           (ffi.istype(interval_t, o) or
+            ffi.istype(interval_months_t, o) or
+            ffi.istype(interval_years_t, o))
 end
 
 local function is_datetime(o)
@@ -189,6 +214,16 @@ local function interval_new()
     return interval
 end
 
+local function check_number(n, message, lvl)
+    if lvl == nil then
+        lvl = 2
+    end
+    if type(n) ~= 'number' then
+        return error(("%s: expected number, but received %s"):
+                     format(message, n), lvl)
+    end
+end
+
 local function check_date(o, message, lvl)
     if lvl == nil then
         lvl = 2
@@ -229,6 +264,56 @@ local function check_str(s, message, lvl)
     end
 end
 
+local function interval_years_new(y)
+    check_number(y, "years(number)")
+    local o = ffi.new(interval_years_t)
+    o.y = y
+    return o
+end
+
+local function interval_months_new(m)
+    check_number(m, "months(number)")
+    local o = ffi.new(interval_months_t)
+    o.m = m
+    return o
+end
+
+local function interval_weeks_new(w)
+    check_number(w, "weeks(number)")
+    local o = ffi.new(interval_t)
+    o.secs = w * SECS_PER_DAY * 7
+    return o
+end
+
+local function interval_days_new(d)
+    check_number(d, "days(number)")
+    local o = ffi.new(interval_t)
+    o.secs = d * SECS_PER_DAY
+    return o
+end
+
+local function interval_hours_new(h)
+    check_number(h, "hours(number)")
+    local o = ffi.new(interval_t)
+    o.secs = h * 60 * 60
+    return o
+end
+
+local function interval_minutes_new(m)
+    check_number(m, "minutes(number)")
+    local o = ffi.new(interval_t)
+    o.secs = m * 60
+    return o
+end
+
+local function interval_seconds_new(s)
+    check_number(s, "seconds(number)")
+    local o = ffi.new(interval_t)
+    o.nsec = s % 1 * 1e9
+    o.secs = s - (s % 1)
+    return o
+end
+
 local function datetime_cmp(lhs, rhs)
     if not is_date_interval(lhs) or
        not is_date_interval(rhs) then
@@ -290,6 +375,88 @@ local function check_range(v, range, txt)
     end
 end
 
+-- addition or subtraction from date/time of a given interval
+-- described via table direction should be +1 or -1
+local function interval_increment(self, o, direction)
+    assert(direction == -1 or direction == 1)
+    local title = direction > 0 and "datetime.add" or "datetime.sub"
+    check_date(self, title)
+    if type(o) ~= 'table' then
+        error(('%s - object expected'):format(title), 2)
+    end
+
+    local ym_updated = false
+    local dhms_updated = false
+
+    local dt = local_dt(self)
+    local secs, nsec
+    secs, nsec = self.secs, self.nsec
+
+    local handlers = {
+        years = function(k, v)
+            check_range(v, {0, 9999}, k)
+            dt = builtin.dt_add_years(dt, direction * v, builtin.DT_LIMIT)
+            ym_updated = true
+        end,
+
+        months = function(k, v)
+            check_range(v, {0, 12}, k)
+            dt = builtin.dt_add_months(dt, direction * v, builtin.DT_LIMIT)
+            ym_updated = true
+        end,
+
+        weeks = function(k, v)
+            check_range(v, {0, 52}, k)
+            secs = secs + direction * 7 * v * SECS_PER_DAY
+            dhms_updated = true
+        end,
+
+        days = function(k, v)
+            check_range(v, {0, 31}, k)
+            secs = secs + direction * v * SECS_PER_DAY
+            dhms_updated = true
+        end,
+
+        hours = function(k, v)
+            check_range(v, {0, 23}, k)
+            secs = secs + direction * 60 * 60 * v
+            dhms_updated = true
+        end,
+
+        minutes = function(k, v)
+            check_range(v, {0, 59}, k)
+            secs = secs + direction * 60 * v
+        end,
+
+        seconds = function(k, v)
+            check_range(v, {0, 60}, k)
+            local s, frac = math.modf(v)
+            secs = secs + direction * s
+            nsec = nsec + direction * frac * 1e9
+            dhms_updated = true
+        end,
+    }
+    for key, value in pairs(o) do
+        handlers[key](key, value)
+    end
+
+    secs, nsec = _normalize_nsec(secs, nsec)
+
+    -- .days, .hours, .minutes, .seconds
+    if dhms_updated then
+        self.secs = secs
+        self.nsec = nsec
+    end
+
+    -- .years, .months updated
+    if ym_updated then
+        self.secs = (builtin.dt_rdn(dt) - DT_EPOCH_1970_OFFSET) * SECS_PER_DAY +
+                    secs % SECS_PER_DAY
+    end
+
+    return self
+end
+
 local datetime_index_handlers = {
     unixtime = function(self)
         return self.secs
@@ -326,6 +493,18 @@ local datetime_index_handlers = {
     days = function(self)
         return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60) / 24
     end,
+
+    add = function(self)
+        return function(self, o)
+            return interval_increment(self, o, 1)
+        end
+    end,
+
+    sub = function(self)
+        return function(self, o)
+            return interval_increment(self, o, -1)
+        end
+    end,
 }
 
 local datetime_index = function(self, key)
@@ -480,6 +659,10 @@ local function datetime_tostring(o)
         local len = builtin.datetime_to_string(o, buff, sz)
         assert(len < sz)
         return ffi.string(buff)
+    elseif ffi.typeof(o) == interval_years_t then
+        return ('%+d years'):format(o.y)
+    elseif ffi.typeof(o) == interval_months_t then
+        return ('%+d months'):format(o.m)
     elseif ffi.typeof(o) == interval_t then
         local ts = o.timestamp
         local sign = '+'
@@ -510,12 +693,20 @@ local function date_first(lhs, rhs)
         return rhs, lhs
     end
 end
-
 local function error_incompatible(name)
     error(("datetime:%s() - incompatible type of arguments"):
           format(name), 3)
 end
-
+--[[
+Matrix of subtraction operands eligibility and their result type
+
+|                 |  datetime | interval | interval_months | interval_years |
++-----------------+-----------+----------+-----------------+----------------+
+| datetime        |  interval | datetime | datetime        | datetime       |
+| interval        |           | interval |                 |                |
+| interval_months |           |          | interval_months |                |
+| interval_years  |           |          |                 | interval_years |
+]]
 local function datetime_sub(lhs, rhs)
     check_date_interval(lhs, "operator -")
     local d, s = lhs, rhs
@@ -524,26 +715,53 @@ local function datetime_sub(lhs, rhs)
     local o
 
     if left_t == datetime_t then
-        -- left is date, right is date or generic interval
+        -- 1. left is date, right is date or generic interval
         if (right_t == datetime_t or right_t == interval_t) then
             o = right_t == datetime_t and interval_new() or datetime_new()
             o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
                                             lhs.nsec - rhs.nsec)
             return o
+        -- 2. left is date, right is interval in months
+        elseif right_t == interval_months_t then
+            local dt = builtin.dt_add_months(local_dt(lhs), -rhs.m, builtin.DT_LIMIT)
+            return mk_timestamp(dt, lhs.secs % SECS_PER_DAY,
+                                lhs.nsec, lhs.offset)
+
+        -- 3. left is date, right is interval in years
+        elseif right_t == interval_years_t then
+            local dt = builtin.dt_add_years(local_dt(lhs), -rhs.y, builtin.DT_LIMIT)
+            return mk_timestamp(dt, lhs.secs % SECS_PER_DAY,
+                                lhs.nsec, lhs.offset)
         else
             error_incompatible("operator -")
         end
-    -- both left and right are generic intervals
+    -- 4. both left and right are generic intervals
     elseif left_t == interval_t and right_t == interval_t then
         o = interval_new()
         o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
                                         lhs.nsec - rhs.nsec)
         return o
+    -- 5. both left and right are intervals in months
+    elseif left_t == interval_months_t and right_t == interval_months_t then
+        return interval_months_new(lhs.m - rhs.m)
+    -- 5. both left and right are intervals in years
+    elseif left_t == interval_years_t and right_t == interval_years_t then
+        return interval_years_new(lhs.y - rhs.y)
     else
         error_incompatible("operator -")
     end
 end
 
+--[[
+Matrix of addition operands eligibility and their result type
+
+|                 |  datetime | interval | interval_months | interval_years |
++-----------------+-----------+----------+-----------------+----------------+
+| datetime        |  datetime | datetime | datetime        | datetime       |
+| interval        |  datetime | interval |                 |                |
+| interval_months |  datetime |          | interval_months |                |
+| interval_years  |  datetime |          |                 | interval_years |
+]]
 local function datetime_add(lhs, rhs)
     local d, s = date_first(lhs, rhs)
 
@@ -553,16 +771,33 @@ local function datetime_add(lhs, rhs)
     local right_t = ffi.typeof(s)
     local o
 
-    -- left is date, right is date or interval
+    -- 1. left is date, right is date or interval
     if left_t == datetime_t and right_t == interval_t then
         o = datetime_new()
         o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
         return o
-    -- both left and right are generic intervals
+    -- 2. left is date, right is interval in months
+    elseif left_t == datetime_t and right_t == interval_months_t then
+        local dt = builtin.dt_add_months(local_dt(d), s.m, builtin.DT_LIMIT)
+        local secs = d.secs % SECS_PER_DAY
+        return mk_timestamp(dt, secs, d.nsec, d.offset or 0)
+
+    -- 3. left is date, right is interval in years
+    elseif left_t == datetime_t and right_t == interval_years_t then
+        local dt = builtin.dt_add_years(local_dt(d), s.y, builtin.DT_LIMIT)
+        local secs = d.secs % SECS_PER_DAY
+        return mk_timestamp(dt, secs, d.nsec, d.offset or 0)
+    -- 4. both left and right are generic intervals
     elseif left_t == interval_t and right_t == interval_t then
         o = interval_new()
         o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
         return o
+    -- 5. both left and right are intervals in months
+    elseif left_t == interval_months_t and right_t == interval_months_t then
+        return interval_months_new(d.m + s.m)
+    -- 6. both left and right are intervals in years
+    elseif left_t == interval_years_t and right_t == interval_years_t then
+        return interval_years_new(d.y + s.y)
     else
         error_incompatible("operator +")
     end
@@ -755,6 +990,16 @@ local datetime_mt = {
     __add = datetime_add,
     __index = datetime_index,
     __newindex = datetime_newindex,
+
+    add = function(self, o)
+        self = interval_increment(self, o, 1)
+        return self
+    end,
+
+    sub = function(self, o)
+        self = interval_increment(self, o, -1)
+        return self
+    end
 }
 
 local interval_mt = {
@@ -768,12 +1013,29 @@ local interval_mt = {
     __index = datetime_index,
 }
 
+local interval_tiny_mt = {
+    __tostring = datetime_tostring,
+    __serialize = interval_serialize,
+    __sub = datetime_sub,
+    __add = datetime_add,
+    __index = datetime_index,
+}
+
 ffi.metatype(interval_t, interval_mt)
 ffi.metatype(datetime_t, datetime_mt)
+ffi.metatype(interval_years_t, interval_tiny_mt)
+ffi.metatype(interval_months_t, interval_tiny_mt)
 
 return setmetatable(
     {
         new         = datetime_new,
+        years       = interval_years_new,
+        months      = interval_months_new,
+        weeks       = interval_weeks_new,
+        days        = interval_days_new,
+        hours       = interval_hours_new,
+        minutes     = interval_minutes_new,
+        seconds     = interval_seconds_new,
         interval    = interval_new,
 
         parse       = parse,
diff --git a/test/app-tap/datetime.test.lua b/test/app-tap/datetime.test.lua
index f7541567e..678de7311 100755
--- a/test/app-tap/datetime.test.lua
+++ b/test/app-tap/datetime.test.lua
@@ -6,7 +6,7 @@ local date = require('datetime')
 local ffi = require('ffi')
 
 
-test:plan(6)
+test:plan(10)
 
 test:test("Simple tests for parser", function(test)
     test:plan(2)
@@ -203,4 +203,165 @@ test:test("Parse tiny date into seconds and other parts", function(test)
     test:ok(tiny.hours == 0.00848, "hours")
 end)
 
+test:test("Stringization of dates and intervals", function(test)
+    test:plan(13)
+    local str = '19700101Z'
+    local dt = date(str)
+    test:ok(tostring(dt) == '1970-01-01T00:00Z', ('tostring(%s)'):format(str))
+    test:ok(tostring(date.seconds(12)) == '+12 secs', '+12 seconds')
+    test:ok(tostring(date.seconds(-12)) == '-12 secs', '-12 seconds')
+    test:ok(tostring(date.minutes(12)) == '+12 minutes, 0 seconds', '+12 minutes')
+    test:ok(tostring(date.minutes(-12)) == '-12 minutes, 0 seconds', '-12 minutes')
+    test:ok(tostring(date.hours(12)) == '+12 hours, 0 minutes, 0 seconds',
+            '+12 hours')
+    test:ok(tostring(date.hours(-12)) == '-12 hours, 0 minutes, 0 seconds',
+            '-12 hours')
+    test:ok(tostring(date.days(12)) == '+12 days, 0 hours, 0 minutes, 0 seconds',
+            '+12 days')
+    test:ok(tostring(date.days(-12)) == '-12 days, 0 hours, 0 minutes, 0 seconds',
+            '-12 days')
+    test:ok(tostring(date.months(5)) == '+5 months', '+5 months')
+    test:ok(tostring(date.months(-5)) == '-5 months', '-5 months')
+    test:ok(tostring(date.years(4)) == '+4 years', '+4 years')
+    test:ok(tostring(date.years(-4)) == '-4 years', '-4 years')
+end)
+
+test:test("Time interval operations", function(test)
+    test:plan(12)
+
+    -- check arithmetic with leap dates
+    local T = date('1972-02-29')
+    local M = date.months(2)
+    local Y = date.years(1)
+    test:ok(tostring(T + M) == '1972-04-29T00:00Z', ('T(%s) + M(%s'):format(T, M))
+    test:ok(tostring(T + Y) == '1973-03-01T00:00Z', ('T(%s) + Y(%s'):format(T, Y))
+    test:ok(tostring(T + M + Y) == '1973-04-30T00:00Z',
+            ('T(%s) + M(%s) + Y(%s'):format(T, M, Y))
+    test:ok(tostring(T + Y + M) == '1973-05-01T00:00Z',
+            ('T(%s) + M(%s) + Y(%s'):format(T, M, Y))
+    test:ok(tostring(T:add{years = 1, months = 2}) == '1973-04-30T00:00Z',
+            ('T:add{years=1,months=2}(%s)'):format(T))
+
+    -- check average, not leap dates
+    T = date('1970-01-08')
+    test:ok(tostring(T + M) == '1970-03-08T00:00Z', ('T(%s) + M(%s'):format(T, M))
+    test:ok(tostring(T + Y) == '1971-01-08T00:00Z', ('T(%s) + Y(%s'):format(T, Y))
+    test:ok(tostring(T + M + Y) == '1971-03-08T00:00Z',
+            ('T(%s) + M(%s) + Y(%s'):format(T, M, Y))
+    test:ok(tostring(T + Y + M) == '1971-03-08T00:00Z',
+            ('T(%s) + Y(%s) + M(%s'):format(T, Y, M))
+    test:ok(tostring(T:add{years = 1, months = 2}) == '1971-03-08T00:00Z',
+            ('T:add{years=1,months=2}(%s)'):format(T))
+
+
+    -- subtraction of 2 dates
+    local T2 = date('19700103')
+    local T1 = date('1970-01-01')
+    test:ok(tostring(T2 - T1) == '+2 days, 0 hours, 0 minutes, 0 seconds',
+            ('T2(%s) - T1(%s'):format(T2, T1))
+    test:ok(tostring(T1 - T2) == '-2 days, 0 hours, 0 minutes, 0 seconds',
+            ('T2(%s) - T1(%s'):format(T2, T1))
+end)
+
+local function catchadd(A, B)
+    return pcall(function() return A + B end)
+end
+
+--[[
+Matrix of addition operands eligibility and their result type
+
+|                 |  datetime | interval | interval_months | interval_years |
++-----------------+-----------+----------+-----------------+----------------+
+| datetime        |  datetime | datetime | datetime        | datetime       |
+| interval        |  datetime | interval |                 |                |
+| interval_months |  datetime |          | interval_months |                |
+| interval_years  |  datetime |          |                 | interval_years |
+]]
+
+test:test("Matrix of allowed time and interval additions", function(test)
+    test:plan(20)
+
+    -- check arithmetic with leap dates
+    local T1970 = date('1970-01-01')
+    local T2000 = date('2000-01-01')
+    local I1 = date.days(1)
+    local M2 = date.months(2)
+    local M10 = date.months(10)
+    local Y1 = date.years(1)
+    local Y5 = date.years(5)
+
+    test:ok(catchadd(T1970, I1) == true, "status: T + I")
+    test:ok(catchadd(T1970, M2) == true, "status: T + M")
+    test:ok(catchadd(T1970, Y1) == true, "status: T + Y")
+    test:ok(catchadd(T1970, T2000) == false, "status: T + T")
+    test:ok(catchadd(I1, T1970) == true, "status: I + T")
+    test:ok(catchadd(M2, T1970) == true, "status: M + T")
+    test:ok(catchadd(Y1, T1970) == true, "status: Y + T")
+    test:ok(catchadd(I1, Y1) == false, "status: I + Y")
+    test:ok(catchadd(M2, Y1) == false, "status: M + Y")
+    test:ok(catchadd(I1, Y1) == false, "status: I + Y")
+    test:ok(catchadd(Y5, M10) == false, "status: Y + M")
+    test:ok(catchadd(Y5, I1) == false, "status: Y + I")
+    test:ok(catchadd(Y5, Y1) == true, "status: Y + Y")
+
+    test:ok(tostring(T1970 + I1) == "1970-01-02T00:00Z", "value: T + I")
+    test:ok(tostring(T1970 + M2) == "1970-03-01T00:00Z", "value: T + M")
+    test:ok(tostring(T1970 + Y1) == "1971-01-01T00:00Z", "value: T + Y")
+    test:ok(tostring(I1 + T1970) == "1970-01-02T00:00Z", "value: I + T")
+    test:ok(tostring(M2 + T1970) == "1970-03-01T00:00Z", "value: M + T")
+    test:ok(tostring(Y1 + T1970) == "1971-01-01T00:00Z", "value: Y + T")
+    test:ok(tostring(Y5 + Y1) == "+6 years", "Y + Y")
+
+end)
+
+local function catchsub_status(A, B)
+    return pcall(function() return A - B end)
+end
+
+--[[
+Matrix of subtraction operands eligibility and their result type
+
+|                 |  datetime | interval | interval_months | interval_years |
++-----------------+-----------+----------+-----------------+----------------+
+| datetime        |  interval | datetime | datetime        | datetime       |
+| interval        |           | interval |                 |                |
+| interval_months |           |          | interval_months |                |
+| interval_years  |           |          |                 | interval_years |
+]]
+test:test("Matrix of allowed time and interval subtractions", function(test)
+    test:plan(18)
+
+    -- check arithmetic with leap dates
+    local T1970 = date('1970-01-01')
+    local T2000 = date('2000-01-01')
+    local I1 = date.days(1)
+    local M2 = date.months(2)
+    local M10 = date.months(10)
+    local Y1 = date.years(1)
+    local Y5 = date.years(5)
+
+    test:ok(catchsub_status(T1970, I1) == true, "status: T - I")
+    test:ok(catchsub_status(T1970, M2) == true, "status: T - M")
+    test:ok(catchsub_status(T1970, Y1) == true, "status: T - Y")
+    test:ok(catchsub_status(T1970, T2000) == true, "status: T - T")
+    test:ok(catchsub_status(I1, T1970) == false, "status: I + T")
+    test:ok(catchsub_status(M2, T1970) == false, "status: M + T")
+    test:ok(catchsub_status(Y1, T1970) == false, "status: Y + T")
+    test:ok(catchsub_status(I1, Y1) == false, "status: I - Y")
+    test:ok(catchsub_status(M2, Y1) == false, "status: M - Y")
+    test:ok(catchsub_status(I1, Y1) == false, "status: I - Y")
+    test:ok(catchsub_status(Y5, M10) == false, "status: Y - M")
+    test:ok(catchsub_status(Y5, I1) == false, "status: Y - I")
+    test:ok(catchsub_status(Y5, Y1) == true, "status: Y - Y")
+
+    test:ok(tostring(T1970 - I1) == "1969-12-31T00:00Z", "value: T - I")
+    test:ok(tostring(T1970 - M2) == "1969-11-01T00:00Z", "value: T - M")
+    test:ok(tostring(T1970 - Y1) == "1969-01-01T00:00Z", "value: T - Y")
+    test:ok(tostring(T1970 - T2000) == "-10957 days, 0 hours, 0 minutes, 0 seconds",
+            "value: T - T")
+    test:ok(tostring(Y5 - Y1) == "+4 years", "value: Y - Y")
+
+
+end)
+
 os.exit(test:check() and 0 or 1)
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 8/9] datetime: changelog for datetime module
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (6 preceding siblings ...)
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 7/9] lua, datetime: time intervals support Timur Safin via Tarantool-patches
@ 2021-08-02  0:41 ` Timur Safin via Tarantool-patches
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 9/9] lua, box, datetime: rename struct datetime_t Timur Safin via Tarantool-patches
                   ` (3 subsequent siblings)
  11 siblings, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:41 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

Introduced new date/time/interval types support to lua and storage engines.

Closes #5941
Closes #5946
---
 changelogs/unreleased/gh-5941-datetime-type-support.md | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 changelogs/unreleased/gh-5941-datetime-type-support.md

diff --git a/changelogs/unreleased/gh-5941-datetime-type-support.md b/changelogs/unreleased/gh-5941-datetime-type-support.md
new file mode 100644
index 000000000..3c755008e
--- /dev/null
+++ b/changelogs/unreleased/gh-5941-datetime-type-support.md
@@ -0,0 +1,4 @@
+## feature/lua/datetime
+
+ * Introduce new builtin module for date/time/interval support - `datetime.lua`.
+   Support of new datetime type in storage engines (gh-5941, gh-5946).
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 9/9] lua, box, datetime: rename struct datetime_t
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (7 preceding siblings ...)
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 8/9] datetime: changelog for datetime module Timur Safin via Tarantool-patches
@ 2021-08-02  0:41 ` Timur Safin via Tarantool-patches
  2021-08-06  0:27   ` Safin Timur via Tarantool-patches
  2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 1/2] datetime: update tests for macosx Timur Safin via Tarantool-patches
                   ` (2 subsequent siblings)
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-02  0:41 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

Renamed all references:
- `struct datetime_t` -> `struct datetime`;
- `struct datetime_interval_t` -> `struct datetime_interval`.

Part of #5941
---
 src/box/tuple_compare.cc    | 10 +++++-----
 src/lib/core/datetime.c     | 25 ++++++++++++-------------
 src/lib/core/datetime.h     | 29 ++++++++++++++---------------
 src/lib/mpstream/mpstream.c |  3 +--
 src/lib/mpstream/mpstream.h |  5 ++---
 src/lua/datetime.lua        |  6 +++---
 src/lua/msgpack.c           |  2 +-
 src/lua/msgpackffi.lua      |  8 ++++----
 src/lua/serializer.c        |  2 +-
 src/lua/serializer.h        |  2 +-
 src/lua/utils.c             | 10 +++++-----
 src/lua/utils.h             |  4 ++--
 test/unit/datetime.c        | 17 +++++++++--------
 13 files changed, 60 insertions(+), 63 deletions(-)

diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index f733b9f01..530417fcc 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -397,8 +397,8 @@ mp_compare_uuid(const char *field_a, const char *field_b)
 static int
 mp_compare_datetime(const char *lhs, const char *rhs)
 {
-	datetime_t lhs_dt, rhs_dt;
-	datetime_t *ret;
+	datetime lhs_dt, rhs_dt;
+	datetime *ret;
 	ret = mp_decode_datetime(&lhs, &lhs_dt);
 	assert(ret != NULL);
 	ret = mp_decode_datetime(&rhs, &rhs_dt);
@@ -1648,7 +1648,7 @@ hint_uuid_raw(const char *data)
 }
 
 static inline hint_t
-hint_datetime(struct datetime_t *date)
+hint_datetime(struct datetime *date)
 {
 	/*
 	 * Use at most HINT_VALUE_SECS_BITS from datetime
@@ -1803,7 +1803,7 @@ field_hint_datetime(const char *field)
 	int8_t ext_type;
 	uint32_t len = mp_decode_extl(&field, &ext_type);
 	assert(ext_type == MP_DATETIME);
-	struct datetime_t date;
+	struct datetime date;
 	return hint_datetime(datetime_unpack(&field, len, &date));
 }
 
@@ -1860,7 +1860,7 @@ field_hint_scalar(const char *field, struct coll *coll)
 			return hint_uuid_raw(field);
 		case MP_DATETIME:
 		{
-			struct datetime_t date;
+			struct datetime date;
 			return hint_datetime(datetime_unpack(&field, len, &date));
 		}
 		default:
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
index 96e554bd2..03facb123 100755
--- a/src/lib/core/datetime.c
+++ b/src/lib/core/datetime.c
@@ -84,7 +84,7 @@ mp_decode_Xint(const char **data)
 }
 
 static inline uint32_t
-mp_sizeof_datetime_raw(const struct datetime_t *date)
+mp_sizeof_datetime_raw(const struct datetime *date)
 {
 	uint32_t sz = mp_sizeof_Xint(date->secs);
 
@@ -98,13 +98,13 @@ mp_sizeof_datetime_raw(const struct datetime_t *date)
 }
 
 uint32_t
-mp_sizeof_datetime(const struct datetime_t *date)
+mp_sizeof_datetime(const struct datetime *date)
 {
 	return mp_sizeof_ext(mp_sizeof_datetime_raw(date));
 }
 
-struct datetime_t *
-datetime_unpack(const char **data, uint32_t len, struct datetime_t *date)
+struct datetime *
+datetime_unpack(const char **data, uint32_t len, struct datetime *date)
 {
 	const char * svp = *data;
 
@@ -128,8 +128,8 @@ datetime_unpack(const char **data, uint32_t len, struct datetime_t *date)
 	return date;
 }
 
-struct datetime_t *
-mp_decode_datetime(const char **data, struct datetime_t *date)
+struct datetime *
+mp_decode_datetime(const char **data, struct datetime *date)
 {
 	if (mp_typeof(**data) != MP_EXT)
 		return NULL;
@@ -144,7 +144,7 @@ mp_decode_datetime(const char **data, struct datetime_t *date)
 }
 
 char *
-datetime_pack(char *data, const struct datetime_t *date)
+datetime_pack(char *data, const struct datetime *date)
 {
 	data = mp_encode_Xint(data, date->secs);
 	if (date->nsec != 0 || date->offset != 0)
@@ -156,7 +156,7 @@ datetime_pack(char *data, const struct datetime_t *date)
 }
 
 char *
-mp_encode_datetime(char *data, const struct datetime_t *date)
+mp_encode_datetime(char *data, const struct datetime *date)
 {
 	uint32_t len = mp_sizeof_datetime_raw(date);
 
@@ -165,8 +165,7 @@ mp_encode_datetime(char *data, const struct datetime_t *date)
 	return datetime_pack(data, date);
 }
 
-int
-datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
+int datetime_to_string(const struct datetime *date, char *buf, uint32_t len)
 {
 	char * src = buf;
 	int offset = date->offset;
@@ -218,7 +217,7 @@ datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
 int
 mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
 {
-	struct datetime_t date = {0};
+	struct datetime date = {0};
 
 	if (datetime_unpack(data, len, &date) == NULL)
 		return -1;
@@ -229,7 +228,7 @@ mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
 int
 mp_fprint_datetime(FILE *file, const char **data, uint32_t len)
 {
-	struct  datetime_t date;
+	struct datetime date;
 
 	if (datetime_unpack(data, len, &date) == NULL)
 		return -1;
@@ -240,7 +239,7 @@ mp_fprint_datetime(FILE *file, const char **data, uint32_t len)
 	return fprintf(file, "%s", buf);
 }
 
-int datetime_compare(const struct datetime_t *lhs, const struct datetime_t *rhs)
+int datetime_compare(const struct datetime *lhs, const struct datetime *rhs)
 {
 	int result = COMPARE_RESULT(lhs->secs, rhs->secs);
 	if (result != 0)
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
index 2c9530ad7..540bd68d9 100644
--- a/src/lib/core/datetime.h
+++ b/src/lib/core/datetime.h
@@ -50,7 +50,8 @@ extern "C"
  * Time is kept normalized to UTC, time-zone offset
  * is informative only.
  */
-struct datetime_t {
+struct datetime
+{
 	int64_t secs;	/**< seconds since epoch */
 	int32_t nsec;	/**< nanoseconds if any */
 	int32_t offset; /**< offset in minutes from UTC */
@@ -59,42 +60,41 @@ struct datetime_t {
 /**
  * Date/time interval structure
  */
-struct datetime_interval_t {
+struct datetime_interval
+{
 	int64_t secs; /**< relative seconds delta */
 	int32_t nsec; /**< nanoseconds delta */
 };
 
-int
-datetime_compare(const struct datetime_t * lhs,
-		 const struct datetime_t * rhs);
+int datetime_compare(const struct datetime *lhs,
+		     const struct datetime *rhs);
 
-
-struct datetime_t *
-datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
+struct datetime *
+datetime_unpack(const char **data, uint32_t len, struct datetime *date);
 
 /**
  * Pack datetime_t data to the MessagePack buffer.
  */
 char *
-datetime_pack(char *data, const struct datetime_t *date);
+datetime_pack(char *data, const struct datetime *date);
 
 /**
  * Calculate size of MessagePack buffer for datetime_t data.
  */
 uint32_t
-mp_sizeof_datetime(const struct datetime_t *date);
+mp_sizeof_datetime(const struct datetime *date);
 
 /**
  * Decode data from MessagePack buffer to datetime_t structure.
  */
-struct datetime_t *
-mp_decode_datetime(const char **data, struct datetime_t *date);
+struct datetime *
+mp_decode_datetime(const char **data, struct datetime *date);
 
 /**
  * Encode datetime_t structure to the MessagePack buffer.
  */
 char *
-mp_encode_datetime(char *data, const struct datetime_t *date);
+mp_encode_datetime(char *data, const struct datetime *date);
 
 /**
  * Convert datetime to string using default format
@@ -102,8 +102,7 @@ mp_encode_datetime(char *data, const struct datetime_t *date);
  * @param buf output character buffer
  * @param len size ofoutput buffer
  */
-int
-datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
+int datetime_to_string(const struct datetime *date, char *buf, uint32_t len);
 
 int
 mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
diff --git a/src/lib/mpstream/mpstream.c b/src/lib/mpstream/mpstream.c
index 1077e3b19..2af77c205 100644
--- a/src/lib/mpstream/mpstream.c
+++ b/src/lib/mpstream/mpstream.c
@@ -209,8 +209,7 @@ mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid)
 	mpstream_advance(stream, pos - data);
 }
 
-void
-mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *val)
+void mpstream_encode_datetime(struct mpstream *stream, const struct datetime *val)
 {
 	char *data = mpstream_reserve(stream, mp_sizeof_datetime(val));
 	if (data == NULL)
diff --git a/src/lib/mpstream/mpstream.h b/src/lib/mpstream/mpstream.h
index 540e9a666..9e0ed6e72 100644
--- a/src/lib/mpstream/mpstream.h
+++ b/src/lib/mpstream/mpstream.h
@@ -39,7 +39,7 @@ extern "C" {
 #endif /* defined(__cplusplus) */
 
 struct tt_uuid;
-struct datetime_t;
+struct datetime;
 
 /**
 * Ask the allocator to reserve at least size bytes. It can reserve
@@ -146,8 +146,7 @@ mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val);
 void
 mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid);
 
-void
-mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *dt);
+void mpstream_encode_datetime(struct mpstream *stream, const struct datetime *dt);
 
 /** Copies n bytes from memory area src to stream. */
 void
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
index dc88a9d9d..5cad4e02f 100644
--- a/src/lua/datetime.lua
+++ b/src/lua/datetime.lua
@@ -80,7 +80,7 @@ ffi.cdef [[
     // datetime.c
 
     int
-    datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
+    datetime_to_string(const struct datetime * date, char *buf, uint32_t len);
 
 
     // <asm-generic/posix_types.h>
@@ -180,8 +180,8 @@ local NANOS_PER_SEC    = 1000000000LL
 local DT_EPOCH_1970_OFFSET = 719163LL
 
 
-local datetime_t = ffi.typeof('struct datetime_t')
-local interval_t = ffi.typeof('struct datetime_interval_t')
+local datetime_t = ffi.typeof('struct datetime')
+local interval_t = ffi.typeof('struct datetime_interval')
 ffi.cdef [[
     struct t_interval_months {
         int m;
diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
index 0a4ba8129..9c1acc4dc 100644
--- a/src/lua/msgpack.c
+++ b/src/lua/msgpack.c
@@ -339,7 +339,7 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
 		}
 		case MP_DATETIME:
 		{
-			struct datetime_t * date = luaL_pushdatetime(L);
+			struct datetime *date = luaL_pushdatetime(L);
 			date = datetime_unpack(data, len, date);
 			if (date == NULL)
 				goto ext_decode_err;
diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua
index c47d77acb..fb5e7d644 100644
--- a/src/lua/msgpackffi.lua
+++ b/src/lua/msgpackffi.lua
@@ -40,8 +40,8 @@ decimal_t *
 decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
 struct tt_uuid *
 uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid);
-struct datetime_t *
-datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
+struct datetime *
+datetime_unpack(const char **data, uint32_t len, struct datetime *date);
 ]])
 
 local strict_alignment = (jit.arch == 'arm')
@@ -331,7 +331,7 @@ on_encode(ffi.typeof('float'), encode_float)
 on_encode(ffi.typeof('double'), encode_double)
 on_encode(ffi.typeof('decimal_t'), encode_decimal)
 on_encode(ffi.typeof('struct tt_uuid'), encode_uuid)
-on_encode(ffi.typeof('struct datetime_t'), encode_datetime)
+on_encode(ffi.typeof('struct datetime'), encode_datetime)
 
 --------------------------------------------------------------------------------
 -- Decoder
@@ -527,7 +527,7 @@ local ext_decoder = {
     end,
     -- MP_DATETIME
     [4] = function(data, len)
-        local dt = ffi.new("struct datetime_t")
+        local dt = ffi.new("struct datetime")
         builtin.datetime_unpack(data, len, dt)
         return dt
     end,
diff --git a/src/lua/serializer.c b/src/lua/serializer.c
index c27e62c62..24f4a5ff9 100644
--- a/src/lua/serializer.c
+++ b/src/lua/serializer.c
@@ -547,7 +547,7 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg,
 				field->ext_type = MP_ERROR;
 			} else if (cd->ctypeid == CTID_DATETIME) {
 				field->ext_type = MP_DATETIME;
-				field->dateval = (struct datetime_t*) cdata;
+				field->dateval = (struct datetime *)cdata;
 			} else {
 				field->ext_type = MP_UNKNOWN_EXTENSION;
 			}
diff --git a/src/lua/serializer.h b/src/lua/serializer.h
index 52e51d279..e7a240e0a 100644
--- a/src/lua/serializer.h
+++ b/src/lua/serializer.h
@@ -224,7 +224,7 @@ struct luaL_field {
 		uint32_t size;
 		decimal_t *decval;
 		struct tt_uuid *uuidval;
-		struct datetime_t *dateval;
+		struct datetime *dateval;
 	};
 	enum mp_type type;
 	/* subtypes of MP_EXT */
diff --git a/src/lua/utils.c b/src/lua/utils.c
index 685cab47b..9753016c9 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -110,7 +110,7 @@ luaL_pushuuid(struct lua_State *L)
 	return luaL_pushcdata(L, CTID_UUID);
 }
 
-struct datetime_t *
+struct datetime *
 luaL_pushdatetime(struct lua_State *L)
 {
 	return luaL_pushcdata(L, CTID_DATETIME);
@@ -720,22 +720,22 @@ tarantool_lua_utils_init(struct lua_State *L)
 	CTID_UUID = luaL_ctypeid(L, "struct tt_uuid");
 	assert(CTID_UUID != 0);
 
-	rc = luaL_cdef(L, "struct datetime_t {"
+	rc = luaL_cdef(L, "struct datetime {"
 			  "int64_t secs;"
 			  "int32_t nsec;"
 			  "int32_t offset;"
 			  "};");
 	assert(rc == 0);
 	(void) rc;
-	CTID_DATETIME = luaL_ctypeid(L, "struct datetime_t");
+	CTID_DATETIME = luaL_ctypeid(L, "struct datetime");
 	assert(CTID_DATETIME != 0);
-	rc = luaL_cdef(L, "struct datetime_interval_t {"
+	rc = luaL_cdef(L, "struct datetime_interval {"
 			  "int64_t secs;"
 			  "int32_t nsec;"
 			  "};");
 	assert(rc == 0);
 	(void) rc;
-	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval_t");
+	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval");
 	assert(CTID_INTERVAL != 0);
 
 	lua_pushcfunction(L, luaT_newthread_wrapper);
diff --git a/src/lua/utils.h b/src/lua/utils.h
index bf56b1bb5..c27065a1c 100644
--- a/src/lua/utils.h
+++ b/src/lua/utils.h
@@ -59,7 +59,7 @@ struct lua_State;
 struct ibuf;
 typedef struct ibuf box_ibuf_t;
 struct tt_uuid;
-struct datetime_t;
+struct datetime;
 
 /**
  * Single global lua_State shared by core and modules.
@@ -84,7 +84,7 @@ luaL_pushuuid(struct lua_State *L);
  * @sa luaL_pushcdata
  * @return memory associated with this datetime_t data
  */
-struct datetime_t *
+struct datetime *
 luaL_pushdatetime(struct lua_State *L);
 
 /** \cond public */
diff --git a/test/unit/datetime.c b/test/unit/datetime.c
index 226b6fadb..e8a022542 100644
--- a/test/unit/datetime.c
+++ b/test/unit/datetime.c
@@ -141,26 +141,27 @@ exit:
 #define SECS_PER_DAY      86400
 #define DT_EPOCH_1970_OFFSET 719163
 
-
-struct datetime_t {
+struct datetime
+{
 	int64_t secs;
 	int32_t nsec;
 	int32_t offset;
 };
 
 static int
-local_rd(const struct datetime_t * dt) {
+local_rd(const struct datetime *dt)
+{
 	return (int)(dt->secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET;
 }
 
 static int
-local_dt(const struct datetime_t * dt) {
+local_dt(const struct datetime *dt)
+{
 	return dt_from_rdn(local_rd(dt));
 }
 
-
-struct tm*
-datetime_to_tm(struct datetime_t * dt)
+struct tm *
+datetime_to_tm(struct datetime *dt)
 {
 	static struct tm tm;
 
@@ -198,7 +199,7 @@ static void datetime_test(void)
 		// check that stringized literal produces the same date
 		// time fields
 		static char buff[40];
-		struct datetime_t dt = {secs, nanosecs, ofs};
+		struct datetime dt = {secs, nanosecs, ofs};
 		// datetime_to_tm returns time in GMT zone
 		struct tm * p_tm = datetime_to_tm(&dt);
 		size_t len = strftime(buff, sizeof buff, "%F %T%z", p_tm);
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
@ 2021-08-03 12:02   ` Serge Petrenko via Tarantool-patches
  2021-08-03 12:59     ` Timur Safin via Tarantool-patches
  2021-08-04 10:12     ` Timur Safin via Tarantool-patches
  0 siblings, 2 replies; 38+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-03 12:02 UTC (permalink / raw)
  To: Timur Safin, v.shpilevoy, olegrok; +Cc: tarantool-patches



02.08.2021 03:41, Timur Safin via Tarantool-patches пишет:
> * storage hints implemented for datetime_t values;
> * proper comparison for indices of datetime type.
>
> Part of #5941
> Part of #5946

Hi! Thanks for the patch!

Please find 2 comments below.

> ---
>   src/box/field_def.c           | 18 ++++++++
>   src/box/field_def.h           |  3 ++
>   src/box/memtx_space.c         |  3 +-
>   src/box/tuple_compare.cc      | 56 +++++++++++++++++++++++++
>   src/box/vinyl.c               |  3 +-
>   test/engine/datetime.result   | 77 +++++++++++++++++++++++++++++++++++
>   test/engine/datetime.test.lua | 35 ++++++++++++++++
>   7 files changed, 191 insertions(+), 4 deletions(-)
>   create mode 100644 test/engine/datetime.result
>   create mode 100644 test/engine/datetime.test.lua
>
> diff --git a/src/box/field_def.c b/src/box/field_def.c
> index 6964e3e9f..aaf5f9cff 100644
> --- a/src/box/field_def.c
> +++ b/src/box/field_def.c
> @@ -193,3 +193,21 @@ field_type_by_name(const char *name, size_t len)
>   		return FIELD_TYPE_ANY;
>   	return field_type_MAX;
>   }
> +
> +const bool field_type_index_allowed[] =
> +    {
> +	/* [FIELD_TYPE_ANY]      = */ false,
> +	/* [FIELD_TYPE_UNSIGNED] = */ true,
> +	/* [FIELD_TYPE_STRING]   = */ true,
> +	/* [FIELD_TYPE_NUMBER]   = */ true,
> +	/* [FIELD_TYPE_DOUBLE]   = */ true,
> +	/* [FIELD_TYPE_INTEGER]  = */ true,
> +	/* [FIELD_TYPE_BOOLEAN]  = */ true,
> +	/* [FIELD_TYPE_VARBINARY]= */ true,
> +	/* [FIELD_TYPE_SCALAR]   = */ true,
> +	/* [FIELD_TYPE_DECIMAL]  = */ true,
> +	/* [FIELD_TYPE_UUID]     = */ true,
> +	/* [FIELD_TYPE_ARRAY]    = */ false,
> +	/* [FIELD_TYPE_MAP]      = */ false,
> +	/* [FIELD_TYPE_DATETIME] = */ true,
> +};
> diff --git a/src/box/field_def.h b/src/box/field_def.h
> index 120b2a93d..bd02418df 100644
> --- a/src/box/field_def.h
> +++ b/src/box/field_def.h
> @@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[];
>   extern const struct opt_def field_def_reg[];
>   extern const struct field_def field_def_default;
>   
> +/** helper table for checking allowed indices for types */
> +extern const bool field_type_index_allowed[];
> +
>   /**
>    * @brief Field definition
>    * Contains information about of one tuple field.
> diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c
> index b71318d24..1ab16122e 100644
> --- a/src/box/memtx_space.c
> +++ b/src/box/memtx_space.c
> @@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def)
>   	/* Check that there are no ANY, ARRAY, MAP parts */
>   	for (uint32_t i = 0; i < key_def->part_count; i++) {
>   		struct key_part *part = &key_def->parts[i];
> -		if (part->type <= FIELD_TYPE_ANY ||
> -		    part->type >= FIELD_TYPE_ARRAY) {
> +		if (!field_type_index_allowed[part->type]) {
>   			diag_set(ClientError, ER_MODIFY_INDEX,
>   				 index_def->name, space_name(space),
>   				 tt_sprintf("field type '%s' is not supported",
> diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
> index 48c833643..f733b9f01 100644
> --- a/src/box/tuple_compare.cc
> +++ b/src/box/tuple_compare.cc
> @@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
>   						   field_b, b_type);
>   	case FIELD_TYPE_UUID:
>   		return mp_compare_uuid(field_a, field_b);
> +	case FIELD_TYPE_DATETIME:
> +		return mp_compare_datetime(field_a, field_b);
>   	default:
>   		unreachable();
>   		return 0;
> @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint,
>   #define HINT_VALUE_DOUBLE_MAX	(exp2(HINT_VALUE_BITS - 1) - 1)
>   #define HINT_VALUE_DOUBLE_MIN	(-exp2(HINT_VALUE_BITS - 1))
>   
> +/**
> + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds
> + * into 60 bits of hint value. The idea is to represent wide enough
> + * years range, and leave the rest of bits occupied from nanoseconds part:
> + * - 36 bits is enough for time range of [208BC..4147]
> + * - for nanoseconds there is left 24 bits, which are MSB part of
> + *   32-bit value
> + */
> +#define HINT_VALUE_SECS_BITS	36
> +#define HINT_VALUE_NSEC_BITS	(HINT_VALUE_BITS - HINT_VALUE_SECS_BITS)
> +#define HINT_VALUE_SECS_MAX	((1LL << HINT_VALUE_SECS_BITS) - 1)
> +#define HINT_VALUE_SECS_MIN	(-(1LL << HINT_VALUE_SECS_BITS))
> +#define HINT_VALUE_NSEC_SHIFT	(sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS)
> +#define HINT_VALUE_NSEC_MAX	((1ULL << HINT_VALUE_NSEC_BITS) - 1)
> +
>   /*
>    * HINT_CLASS_BITS should be big enough to store any mp_class value.
>    * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE.
> @@ -1630,6 +1647,24 @@ hint_uuid_raw(const char *data)
>   	return hint_create(MP_CLASS_UUID, val);
>   }
>   
> +static inline hint_t
> +hint_datetime(struct datetime_t *date)
> +{
> +	/*
> +	 * Use at most HINT_VALUE_SECS_BITS from datetime
> +	 * seconds field as a hint value, and at MSB part
> +	 * of HINT_VALUE_NSEC_BITS from nanoseconds.
> +	 */
> +	int64_t secs = date->secs;
> +	int32_t nsec = date->nsec;
> +	uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 :
> +			secs - HINT_VALUE_SECS_MIN;
> +	val &= HINT_VALUE_SECS_MAX;
> +	val <<= HINT_VALUE_NSEC_BITS;
> +	val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX;
> +	return hint_create(MP_CLASS_DATETIME, val);
> +}
> +
I like the idea with having hints for some "near" dates.

You just need to assign the same HINT_VALUE_MAX hint to every datetime value
with date->secs >= HINT_VALUE_SECS_MAX. Otherwise the comparison would
make mistakes (judging by hint values only) for such far away dates.

>   static inline uint64_t
>   hint_str_raw(const char *s, uint32_t len)
>   {
> @@ -1761,6 +1796,17 @@ field_hint_uuid(const char *field)
>   	return hint_uuid_raw(data);
>   }
>   
> +static inline hint_t
> +field_hint_datetime(const char *field)
> +{
> +	assert(mp_typeof(*field) == MP_EXT);
> +	int8_t ext_type;
> +	uint32_t len = mp_decode_extl(&field, &ext_type);
> +	assert(ext_type == MP_DATETIME);
> +	struct datetime_t date;
> +	return hint_datetime(datetime_unpack(&field, len, &date));
> +}
> +
>   static inline hint_t
>   field_hint_string(const char *field, struct coll *coll)
>   {
> @@ -1812,6 +1858,11 @@ field_hint_scalar(const char *field, struct coll *coll)
>   		}
>   		case MP_UUID:
>   			return hint_uuid_raw(field);
> +		case MP_DATETIME:
> +		{
> +			struct datetime_t date;
> +			return hint_datetime(datetime_unpack(&field, len, &date));
> +		}

But you don't allow datetime in SCALAR fields, as I see in the previous 
commit.
So you don't need to account for datetime in scalar hints.

>   		default:
>   			unreachable();
>   		}
> @@ -1849,6 +1900,8 @@ field_hint(const char *field, struct coll *coll)
>   		return field_hint_decimal(field);
>   	case FIELD_TYPE_UUID:
>   		return field_hint_uuid(field);
> +	case FIELD_TYPE_DATETIME:
> +		return field_hint_datetime(field);
>   	default:
>   		unreachable();
>   	}
> @@ -1963,6 +2016,9 @@ key_def_set_hint_func(struct key_def *def)
>   	case FIELD_TYPE_UUID:
>   		key_def_set_hint_func<FIELD_TYPE_UUID>(def);
>   		break;
> +	case FIELD_TYPE_DATETIME:
> +		key_def_set_hint_func<FIELD_TYPE_DATETIME>(def);
> +		break;
>   	default:
>   		/* Invalid key definition. */
>   		def->key_hint = NULL;
> diff --git a/src/box/vinyl.c b/src/box/vinyl.c
> index c80b2d99b..360d1fa70 100644
> --- a/src/box/vinyl.c
> +++ b/src/box/vinyl.c
> @@ -662,8 +662,7 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def)
>   	/* Check that there are no ANY, ARRAY, MAP parts */
>   	for (uint32_t i = 0; i < key_def->part_count; i++) {
>   		struct key_part *part = &key_def->parts[i];
> -		if (part->type <= FIELD_TYPE_ANY ||
> -		    part->type >= FIELD_TYPE_ARRAY) {
> +		if (!field_type_index_allowed[part->type]) {
>   			diag_set(ClientError, ER_MODIFY_INDEX,
>   				 index_def->name, space_name(space),
>   				 tt_sprintf("field type '%s' is not supported",
> diff --git a/test/engine/datetime.result b/test/engine/datetime.result
> new file mode 100644
> index 000000000..848a0aaec
> --- /dev/null
> +++ b/test/engine/datetime.result
> @@ -0,0 +1,77 @@
> +-- test-run result file version 2
> +env = require('test_run')
> + | ---
> + | ...
> +test_run = env.new()
> + | ---
> + | ...
> +engine = test_run:get_cfg('engine')
> + | ---
> + | ...
> +
> +date = require('datetime')
> + | ---
> + | ...
> +
> +_ = box.schema.space.create('T', {engine = engine})
> + | ---
> + | ...
> +_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
> + | ---
> + | ...
> +
> +box.space.T:insert{date('1970-01-01')}\
> +box.space.T:insert{date('1970-01-02')}\
> +box.space.T:insert{date('1970-01-03')}\
> +box.space.T:insert{date('2000-01-01')}
> + | ---
> + | ...
> +
> +o = box.space.T:select{}
> + | ---
> + | ...
> +assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
> + | ---
> + | - true
> + | ...
> +assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
> + | ---
> + | - true
> + | ...
> +assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
> + | ---
> + | - true
> + | ...
> +assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
> + | ---
> + | - true
> + | ...
> +
> +for i = 1,16 do\
> +    box.space.T:insert{date.now()}\
> +end
> + | ---
> + | ...
> +
> +a = box.space.T:select{}
> + | ---
> + | ...
> +err = {}
> + | ---
> + | ...
> +for i = 1, #a - 1 do\
> +    if a[i][1] >= a[i+1][1] then\
> +        table.insert(err, {a[i][1], a[i+1][1]})\
> +        break\
> +    end\
> +end
> + | ---
> + | ...
> +
> +err
> + | ---
> + | - []
> + | ...
> +box.space.T:drop()
> + | ---
> + | ...
> diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua
> new file mode 100644
> index 000000000..3685e4d4b
> --- /dev/null
> +++ b/test/engine/datetime.test.lua
> @@ -0,0 +1,35 @@
> +env = require('test_run')
> +test_run = env.new()
> +engine = test_run:get_cfg('engine')
> +
> +date = require('datetime')
> +
> +_ = box.schema.space.create('T', {engine = engine})
> +_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
> +
> +box.space.T:insert{date('1970-01-01')}\
> +box.space.T:insert{date('1970-01-02')}\
> +box.space.T:insert{date('1970-01-03')}\
> +box.space.T:insert{date('2000-01-01')}
> +
> +o = box.space.T:select{}
> +assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
> +assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
> +assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
> +assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
> +
> +for i = 1,16 do\
> +    box.space.T:insert{date.now()}\
> +end
> +
> +a = box.space.T:select{}
> +err = {}
> +for i = 1, #a - 1 do\
> +    if a[i][1] >= a[i+1][1] then\
> +        table.insert(err, {a[i][1], a[i+1][1]})\
> +        break\
> +    end\
> +end
> +
> +err
> +box.space.T:drop()

-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices
  2021-08-03 12:02   ` Serge Petrenko via Tarantool-patches
@ 2021-08-03 12:59     ` Timur Safin via Tarantool-patches
  2021-08-04 10:12     ` Timur Safin via Tarantool-patches
  1 sibling, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-03 12:59 UTC (permalink / raw)
  To: 'Serge Petrenko', v.shpilevoy, olegrok; +Cc: tarantool-patches



> From: Serge Petrenko <sergepetrenko@tarantool.org>
> Subject: Re: [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime
> comparison for indices
> 
> 
> 
> 02.08.2021 03:41, Timur Safin via Tarantool-patches пишет:
> > * storage hints implemented for datetime_t values;
> > * proper comparison for indices of datetime type.
> >
> > Part of #5941
> > Part of #5946
> 
> Hi! Thanks for the patch!
> 
> Please find 2 comments below.
> 
...
> > @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple,
> hint_t tuple_hint,
> >   #define HINT_VALUE_DOUBLE_MAX	(exp2(HINT_VALUE_BITS - 1) - 1)
> >   #define HINT_VALUE_DOUBLE_MIN	(-exp2(HINT_VALUE_BITS - 1))
> >
> > +/**
> > + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds
> > + * into 60 bits of hint value. The idea is to represent wide enough
> > + * years range, and leave the rest of bits occupied from nanoseconds
> part:
> > + * - 36 bits is enough for time range of [208BC..4147]
> > + * - for nanoseconds there is left 24 bits, which are MSB part of
> > + *   32-bit value
> > + */
> > +#define HINT_VALUE_SECS_BITS	36
> > +#define HINT_VALUE_NSEC_BITS	(HINT_VALUE_BITS - HINT_VALUE_SECS_BITS)
> > +#define HINT_VALUE_SECS_MAX	((1LL << HINT_VALUE_SECS_BITS) - 1)
> > +#define HINT_VALUE_SECS_MIN	(-(1LL << HINT_VALUE_SECS_BITS))
> > +#define HINT_VALUE_NSEC_SHIFT	(sizeof(int) * CHAR_BIT -
> HINT_VALUE_NSEC_BITS)
> > +#define HINT_VALUE_NSEC_MAX	((1ULL << HINT_VALUE_NSEC_BITS) - 1)
> > +
> >   /*
> >    * HINT_CLASS_BITS should be big enough to store any mp_class value.
> >    * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE.
> > @@ -1630,6 +1647,24 @@ hint_uuid_raw(const char *data)
> >   	return hint_create(MP_CLASS_UUID, val);
> >   }
> >
> > +static inline hint_t
> > +hint_datetime(struct datetime_t *date)
> > +{
> > +	/*
> > +	 * Use at most HINT_VALUE_SECS_BITS from datetime
> > +	 * seconds field as a hint value, and at MSB part
> > +	 * of HINT_VALUE_NSEC_BITS from nanoseconds.
> > +	 */
> > +	int64_t secs = date->secs;
> > +	int32_t nsec = date->nsec;
> > +	uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 :
> > +			secs - HINT_VALUE_SECS_MIN;
> > +	val &= HINT_VALUE_SECS_MAX;
> > +	val <<= HINT_VALUE_NSEC_BITS;
> > +	val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX;
> > +	return hint_create(MP_CLASS_DATETIME, val);
> > +}
> > +
> I like the idea with having hints for some "near" dates.
> 
> You just need to assign the same HINT_VALUE_MAX hint to every datetime value
> with date->secs >= HINT_VALUE_SECS_MAX. Otherwise the comparison would
> make mistakes (judging by hint values only) for such far away dates.

Yes, thanks for the note, will make sure that values which exceed selected
36-bit will stick to the maximum value.

> 
> >   static inline uint64_t
> >   hint_str_raw(const char *s, uint32_t len)
> >   {


> > @@ -1812,6 +1858,11 @@ field_hint_scalar(const char *field, struct coll
> *coll)
> >   		}
> >   		case MP_UUID:
> >   			return hint_uuid_raw(field);
> > +		case MP_DATETIME:
> > +		{
> > +			struct datetime_t date;
> > +			return hint_datetime(datetime_unpack(&field, len, &date));
> > +		}
> 
> But you don't allow datetime in SCALAR fields, as I see in the previous
> commit.
> So you don't need to account for datetime in scalar hints.

That's bug - every type is scalar type, unless it's collection (map or array)
so datetime should be considered as scalar. That was oversight if not. 
Thanks!

> 
> >   		default:
> >   			unreachable();
> >   		}
 
> --
> Serge Petrenko

Thanks,
Timur


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
@ 2021-08-03 13:38   ` Timur Safin via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-03 13:38 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

After discussions with Sergey Petrenko I've realized
that there were several bugs in a way datetime was introduced
to fied_def.c, please see patch below...


> From: Timur Safin <tsafin@tarantool.org>
> Subject: [PATCH v3 5/9] box, datetime: messagepack support for datetime
> 
> Serialize datetime_t as newly introduced MP_EXT type.
> It saves 1 required integer field and upto 2 optional
> unsigned fields in very compact fashion.
> - secs is required field;
> - but nsec, offset are both optional;
> 
> * json, yaml serialization formats, lua output mode
>   supported;
> * exported symbols for datetime messagepack size calculations
>   so they are available for usage on Lua side.
> 
> Part of #5941
> Part of #5946


> 
> diff --git a/src/box/field_def.c b/src/box/field_def.c
> index 51acb8025..6964e3e9f 100644
> --- a/src/box/field_def.c
> +++ b/src/box/field_def.c
> @@ -67,11 +67,12 @@ const uint32_t field_mp_type[] = {
>  	/* [FIELD_TYPE_VARBINARY] =  */ 1U << MP_BIN,
>  	/* [FIELD_TYPE_SCALAR]   =  */ (1U << MP_UINT) | (1U << MP_INT) |
>  		(1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_STR) |
> -		(1U << MP_BIN) | (1U << MP_BOOL),
> +		(1U << MP_BIN) | (1U << MP_BOOL) | (1U << MP_DATETIME),
>  	/* [FIELD_TYPE_DECIMAL]  =  */ 0, /* only MP_DECIMAL is supported */
>  	/* [FIELD_TYPE_UUID]     =  */ 0, /* only MP_UUID is supported */
>  	/* [FIELD_TYPE_ARRAY]    =  */ 1U << MP_ARRAY,
>  	/* [FIELD_TYPE_MAP]      =  */ (1U << MP_MAP),
> +	/* [FIELD_TYPE_DATETIME] =  */ 0, /* only MP_DATETIME is supported */
>  };

Here datetime is in the incorrect definition: it's in field_mp_type[]
but should be in field_ext_type[] as bein MP_EXT type.

> 
>  const uint32_t field_ext_type[] = {
> @@ -88,6 +89,7 @@ const uint32_t field_ext_type[] = {
>  	/* [FIELD_TYPE_UUID]      = */ 1U << MP_UUID,
>  	/* [FIELD_TYPE_ARRAY]     = */ 0,
>  	/* [FIELD_TYPE_MAP]       = */ 0,
> +	/* [FIELD_TYPE_DATETIME]  = */ 1U << MP_DATETIME,
>  };
> 
>  const char *field_type_strs[] = {
> @@ -104,6 +106,7 @@ const char *field_type_strs[] = {
>  	/* [FIELD_TYPE_UUID]     = */ "uuid",
>  	/* [FIELD_TYPE_ARRAY]    = */ "array",
>  	/* [FIELD_TYPE_MAP]      = */ "map",
> +	/* [FIELD_TYPE_DATETIME] = */ "datetime",
>  };
> 
>  const char *on_conflict_action_strs[] = {
> @@ -128,20 +131,21 @@ field_type_by_name_wrapper(const char *str, uint32_t
> len)
>   * values can be stored in the j type.
>   */
>  static const bool field_type_compatibility[] = {
> -	   /*   ANY   UNSIGNED  STRING   NUMBER  DOUBLE  INTEGER  BOOLEAN
> VARBINARY SCALAR  DECIMAL   UUID    ARRAY    MAP  */
> -/*   ANY    */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   false,   false,
> -/* UNSIGNED */ true,   true,    false,   true,    false,   true,    false,
> false,  true,   false,  false,   false,   false,
> -/*  STRING  */ true,   false,   true,    false,   false,   false,   false,
> false,  true,   false,  false,   false,   false,
> -/*  NUMBER  */ true,   false,   false,   true,    false,   false,   false,
> false,  true,   false,  false,   false,   false,
> -/*  DOUBLE  */ true,   false,   false,   true,    true,    false,   false,
> false,  true,   false,  false,   false,   false,
> -/*  INTEGER */ true,   false,   false,   true,    false,   true,    false,
> false,  true,   false,  false,   false,   false,
> -/*  BOOLEAN */ true,   false,   false,   false,   false,   false,   true,
> false,  true,   false,  false,   false,   false,
> -/* VARBINARY*/ true,   false,   false,   false,   false,   false,   false,
> true,   true,   false,  false,   false,   false,
> -/*  SCALAR  */ true,   false,   false,   false,   false,   false,   false,
> false,  true,   false,  false,   false,   false,
> -/*  DECIMAL */ true,   false,   false,   true,    false,   false,   false,
> false,  true,   true,   false,   false,   false,
> -/*   UUID   */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  true,    false,   false,
> -/*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   true,    false,
> -/*    MAP   */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   false,   true,
> +	   /*   ANY   UNSIGNED  STRING   NUMBER  DOUBLE  INTEGER  BOOLEAN
> VARBINARY SCALAR  DECIMAL   UUID    ARRAY    MAP     DATETIME */
> +/*   ANY    */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   false,   false,   false,
> +/* UNSIGNED */ true,   true,    false,   true,    false,   true,    false,
> false,  true,   false,  false,   false,   false,   false,
> +/*  STRING  */ true,   false,   true,    false,   false,   false,   false,
> false,  true,   false,  false,   false,   false,   false,
> +/*  NUMBER  */ true,   false,   false,   true,    false,   false,   false,
> false,  true,   false,  false,   false,   false,   false,
> +/*  DOUBLE  */ true,   false,   false,   true,    true,    false,   false,
> false,  true,   false,  false,   false,   false,   false,
> +/*  INTEGER */ true,   false,   false,   true,    false,   true,    false,
> false,  true,   false,  false,   false,   false,   false,
> +/*  BOOLEAN */ true,   false,   false,   false,   false,   false,   true,
> false,  true,   false,  false,   false,   false,   false,
> +/* VARBINARY*/ true,   false,   false,   false,   false,   false,   false,
> true,   true,   false,  false,   false,   false,   false,
> +/*  SCALAR  */ true,   false,   false,   false,   false,   false,   false,
> false,  true,   false,  false,   false,   false,   false,
> +/*  DECIMAL */ true,   false,   false,   true,    false,   false,   false,
> false,  true,   true,   false,   false,   false,   false,
> +/*   UUID   */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  true,    false,   false,   false,
> +/*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   true,    false,   false,
> +/*    MAP   */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   false,   true,    false,
> +/* DATETIME */ true,   false,   false,   false,   false,   false,   false,
> false,  false,  false,  false,   false,   false,   true,
>  };

Also here we did not put datetime to the scalar column, while as 
being scalar type it should be possible to assign datetime values
to the scalar field. [Also we may pay attention to the fact that
uuid has the same problem - it will be investigated separately]

Here are changes we need to introduce to the code in patch:

------------------------------------------
diff --git a/src/box/field_def.c b/src/box/field_def.c
index 6964e3e9f..2682a42ee 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -67,7 +67,7 @@ const uint32_t field_mp_type[] = {
        /* [FIELD_TYPE_VARBINARY] =  */ 1U << MP_BIN,
        /* [FIELD_TYPE_SCALAR]   =  */ (1U << MP_UINT) | (1U << MP_INT) |
                (1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_STR) |
-               (1U << MP_BIN) | (1U << MP_BOOL) | (1U << MP_DATETIME),
+               (1U << MP_BIN) | (1U << MP_BOOL),
        /* [FIELD_TYPE_DECIMAL]  =  */ 0, /* only MP_DECIMAL is supported */
        /* [FIELD_TYPE_UUID]     =  */ 0, /* only MP_UUID is supported */
        /* [FIELD_TYPE_ARRAY]    =  */ 1U << MP_ARRAY,
@@ -84,7 +84,8 @@ const uint32_t field_ext_type[] = {
        /* [FIELD_TYPE_INTEGER]   = */ 0,
        /* [FIELD_TYPE_BOOLEAN]   = */ 0,
        /* [FIELD_TYPE_VARBINARY] = */ 0,
-       /* [FIELD_TYPE_SCALAR]    = */ (1U << MP_DECIMAL) | (1U << MP_UUID),
+       /* [FIELD_TYPE_SCALAR]    = */ (1U << MP_DECIMAL) | (1U << MP_UUID) |
+               (1U << MP_DATETIME),
        /* [FIELD_TYPE_DECIMAL]   = */ 1U << MP_DECIMAL,
        /* [FIELD_TYPE_UUID]      = */ 1U << MP_UUID,
        /* [FIELD_TYPE_ARRAY]     = */ 0,
@@ -145,7 +146,7 @@ static const bool field_type_compatibility[] = {
 /*   UUID   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  true,    false,   false,   false,
 /*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   true,    false,   false,
 /*    MAP   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   true,    false,
-/* DATETIME */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   false,   true,
+/* DATETIME */ true,   false,   false,   false,   false,   false,   false,   false,  true,   false,  false,   false,   false,   true,
 };

 Bool
------------------------------------------

Will update patch to push it very soon.

Regards,
Timur


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 1/2] datetime: update tests for macosx
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (8 preceding siblings ...)
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 9/9] lua, box, datetime: rename struct datetime_t Timur Safin via Tarantool-patches
@ 2021-08-03 21:23 ` Timur Safin via Tarantool-patches
  2021-08-06  0:28   ` Safin Timur via Tarantool-patches
  2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers Timur Safin via Tarantool-patches
  2021-08-03 21:26 ` [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-03 21:23 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

`strftime()` `%z` outputs local time-zone if `tm_gmtoff` is 0.
This behaviour is different to that we observe on Linux, thus we
have different execution results. Updated test to not use `%z`, and only
operate with normalized date time formats `%F` and `%T`

Part of #5941
---
 test/unit/datetime.c      |   2 +-
 test/unit/datetime.result | 284 +++++++++++++++++++-------------------
 2 files changed, 143 insertions(+), 143 deletions(-)

diff --git a/test/unit/datetime.c b/test/unit/datetime.c
index e8a022542..95df7ce68 100644
--- a/test/unit/datetime.c
+++ b/test/unit/datetime.c
@@ -202,7 +202,7 @@ static void datetime_test(void)
 		struct datetime dt = {secs, nanosecs, ofs};
 		// datetime_to_tm returns time in GMT zone
 		struct tm * p_tm = datetime_to_tm(&dt);
-		size_t len = strftime(buff, sizeof buff, "%F %T%z", p_tm);
+		size_t len = strftime(buff, sizeof buff, "%F %T", p_tm);
 		ok(len > 0, "strftime");
 		rc = parse_datetime(buff, len, &dt.secs, &dt.nsec, &dt.offset);
 		is(rc, 0, "correct parse_datetime return value for '%s'", buff);
diff --git a/test/unit/datetime.result b/test/unit/datetime.result
index f106fa769..33997d9df 100644
--- a/test/unit/datetime.result
+++ b/test/unit/datetime.result
@@ -3,356 +3,356 @@
     ok 1 - correct parse_datetime return value for '2012-12-24 15:30Z'
     ok 2 - correct parse_datetime output seconds for '2012-12-24 15:30Z
     ok 3 - strftime
-    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 6 - correct parse_datetime return value for '2012-12-24 15:30z'
     ok 7 - correct parse_datetime output seconds for '2012-12-24 15:30z
     ok 8 - strftime
-    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 11 - correct parse_datetime return value for '2012-12-24 15:30'
     ok 12 - correct parse_datetime output seconds for '2012-12-24 15:30
     ok 13 - strftime
-    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 16 - correct parse_datetime return value for '2012-12-24 16:30+01:00'
     ok 17 - correct parse_datetime output seconds for '2012-12-24 16:30+01:00
     ok 18 - strftime
-    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 21 - correct parse_datetime return value for '2012-12-24 16:30+0100'
     ok 22 - correct parse_datetime output seconds for '2012-12-24 16:30+0100
     ok 23 - strftime
-    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 26 - correct parse_datetime return value for '2012-12-24 16:30+01'
     ok 27 - correct parse_datetime output seconds for '2012-12-24 16:30+01
     ok 28 - strftime
-    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 31 - correct parse_datetime return value for '2012-12-24 14:30-01:00'
     ok 32 - correct parse_datetime output seconds for '2012-12-24 14:30-01:00
     ok 33 - strftime
-    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 36 - correct parse_datetime return value for '2012-12-24 14:30-0100'
     ok 37 - correct parse_datetime output seconds for '2012-12-24 14:30-0100
     ok 38 - strftime
-    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 41 - correct parse_datetime return value for '2012-12-24 14:30-01'
     ok 42 - correct parse_datetime output seconds for '2012-12-24 14:30-01
     ok 43 - strftime
-    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 46 - correct parse_datetime return value for '2012-12-24 15:30:00Z'
     ok 47 - correct parse_datetime output seconds for '2012-12-24 15:30:00Z
     ok 48 - strftime
-    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 51 - correct parse_datetime return value for '2012-12-24 15:30:00z'
     ok 52 - correct parse_datetime output seconds for '2012-12-24 15:30:00z
     ok 53 - strftime
-    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 56 - correct parse_datetime return value for '2012-12-24 15:30:00'
     ok 57 - correct parse_datetime output seconds for '2012-12-24 15:30:00
     ok 58 - strftime
-    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 61 - correct parse_datetime return value for '2012-12-24 16:30:00+01:00'
     ok 62 - correct parse_datetime output seconds for '2012-12-24 16:30:00+01:00
     ok 63 - strftime
-    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 66 - correct parse_datetime return value for '2012-12-24 16:30:00+0100'
     ok 67 - correct parse_datetime output seconds for '2012-12-24 16:30:00+0100
     ok 68 - strftime
-    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 71 - correct parse_datetime return value for '2012-12-24 14:30:00-01:00'
     ok 72 - correct parse_datetime output seconds for '2012-12-24 14:30:00-01:00
     ok 73 - strftime
-    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 76 - correct parse_datetime return value for '2012-12-24 14:30:00-0100'
     ok 77 - correct parse_datetime output seconds for '2012-12-24 14:30:00-0100
     ok 78 - strftime
-    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 81 - correct parse_datetime return value for '2012-12-24 15:30:00.123456Z'
     ok 82 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456Z
     ok 83 - strftime
-    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 86 - correct parse_datetime return value for '2012-12-24 15:30:00.123456z'
     ok 87 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456z
     ok 88 - strftime
-    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 91 - correct parse_datetime return value for '2012-12-24 15:30:00.123456'
     ok 92 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456
     ok 93 - strftime
-    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 96 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01:00'
     ok 97 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01:00
     ok 98 - strftime
-    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 101 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01'
     ok 102 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01
     ok 103 - strftime
-    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 106 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01:00'
     ok 107 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01:00
     ok 108 - strftime
-    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 111 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01'
     ok 112 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01
     ok 113 - strftime
-    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 116 - correct parse_datetime return value for '2012-12-24t15:30Z'
     ok 117 - correct parse_datetime output seconds for '2012-12-24t15:30Z
     ok 118 - strftime
-    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 121 - correct parse_datetime return value for '2012-12-24t15:30z'
     ok 122 - correct parse_datetime output seconds for '2012-12-24t15:30z
     ok 123 - strftime
-    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 126 - correct parse_datetime return value for '2012-12-24t15:30'
     ok 127 - correct parse_datetime output seconds for '2012-12-24t15:30
     ok 128 - strftime
-    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 131 - correct parse_datetime return value for '2012-12-24t16:30+01:00'
     ok 132 - correct parse_datetime output seconds for '2012-12-24t16:30+01:00
     ok 133 - strftime
-    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 136 - correct parse_datetime return value for '2012-12-24t16:30+0100'
     ok 137 - correct parse_datetime output seconds for '2012-12-24t16:30+0100
     ok 138 - strftime
-    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 141 - correct parse_datetime return value for '2012-12-24t14:30-01:00'
     ok 142 - correct parse_datetime output seconds for '2012-12-24t14:30-01:00
     ok 143 - strftime
-    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 146 - correct parse_datetime return value for '2012-12-24t14:30-0100'
     ok 147 - correct parse_datetime output seconds for '2012-12-24t14:30-0100
     ok 148 - strftime
-    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 151 - correct parse_datetime return value for '2012-12-24t15:30:00Z'
     ok 152 - correct parse_datetime output seconds for '2012-12-24t15:30:00Z
     ok 153 - strftime
-    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 156 - correct parse_datetime return value for '2012-12-24t15:30:00z'
     ok 157 - correct parse_datetime output seconds for '2012-12-24t15:30:00z
     ok 158 - strftime
-    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 161 - correct parse_datetime return value for '2012-12-24t15:30:00'
     ok 162 - correct parse_datetime output seconds for '2012-12-24t15:30:00
     ok 163 - strftime
-    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 166 - correct parse_datetime return value for '2012-12-24t16:30:00+01:00'
     ok 167 - correct parse_datetime output seconds for '2012-12-24t16:30:00+01:00
     ok 168 - strftime
-    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 171 - correct parse_datetime return value for '2012-12-24t16:30:00+0100'
     ok 172 - correct parse_datetime output seconds for '2012-12-24t16:30:00+0100
     ok 173 - strftime
-    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 176 - correct parse_datetime return value for '2012-12-24t14:30:00-01:00'
     ok 177 - correct parse_datetime output seconds for '2012-12-24t14:30:00-01:00
     ok 178 - strftime
-    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 181 - correct parse_datetime return value for '2012-12-24t14:30:00-0100'
     ok 182 - correct parse_datetime output seconds for '2012-12-24t14:30:00-0100
     ok 183 - strftime
-    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 186 - correct parse_datetime return value for '2012-12-24t15:30:00.123456Z'
     ok 187 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456Z
     ok 188 - strftime
-    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 191 - correct parse_datetime return value for '2012-12-24t15:30:00.123456z'
     ok 192 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456z
     ok 193 - strftime
-    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 196 - correct parse_datetime return value for '2012-12-24t16:30:00.123456+01:00'
     ok 197 - correct parse_datetime output seconds for '2012-12-24t16:30:00.123456+01:00
     ok 198 - strftime
-    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 201 - correct parse_datetime return value for '2012-12-24t14:30:00.123456-01:00'
     ok 202 - correct parse_datetime output seconds for '2012-12-24t14:30:00.123456-01:00
     ok 203 - strftime
-    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 206 - correct parse_datetime return value for '2012-12-24 16:30 +01:00'
     ok 207 - correct parse_datetime output seconds for '2012-12-24 16:30 +01:00
     ok 208 - strftime
-    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 211 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
     ok 212 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
     ok 213 - strftime
-    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 216 - correct parse_datetime return value for '2012-12-24 15:30 UTC'
     ok 217 - correct parse_datetime output seconds for '2012-12-24 15:30 UTC
     ok 218 - strftime
-    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 221 - correct parse_datetime return value for '2012-12-24 16:30 UTC+1'
     ok 222 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+1
     ok 223 - strftime
-    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 226 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01'
     ok 227 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01
     ok 228 - strftime
-    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 231 - correct parse_datetime return value for '2012-12-24 16:30 UTC+0100'
     ok 232 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+0100
     ok 233 - strftime
-    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 236 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01:00'
     ok 237 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01:00
     ok 238 - strftime
-    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 241 - correct parse_datetime return value for '2012-12-24 14:30 UTC-1'
     ok 242 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-1
     ok 243 - strftime
-    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 246 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01'
     ok 247 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01
     ok 248 - strftime
-    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 251 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01:00'
     ok 252 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01:00
     ok 253 - strftime
-    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 256 - correct parse_datetime return value for '2012-12-24 14:30 UTC-0100'
     ok 257 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-0100
     ok 258 - strftime
-    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 261 - correct parse_datetime return value for '2012-12-24 15:30 GMT'
     ok 262 - correct parse_datetime output seconds for '2012-12-24 15:30 GMT
     ok 263 - strftime
-    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 266 - correct parse_datetime return value for '2012-12-24 16:30 GMT+1'
     ok 267 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+1
     ok 268 - strftime
-    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 271 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01'
     ok 272 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01
     ok 273 - strftime
-    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 276 - correct parse_datetime return value for '2012-12-24 16:30 GMT+0100'
     ok 277 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+0100
     ok 278 - strftime
-    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 281 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01:00'
     ok 282 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01:00
     ok 283 - strftime
-    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 286 - correct parse_datetime return value for '2012-12-24 14:30 GMT-1'
     ok 287 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-1
     ok 288 - strftime
-    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 291 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01'
     ok 292 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01
     ok 293 - strftime
-    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 296 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01:00'
     ok 297 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01:00
     ok 298 - strftime
-    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 301 - correct parse_datetime return value for '2012-12-24 14:30 GMT-0100'
     ok 302 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-0100
     ok 303 - strftime
-    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 306 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
     ok 307 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
     ok 308 - strftime
-    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 311 - correct parse_datetime return value for '2012-12-24 16:30:00 +01:00'
     ok 312 - correct parse_datetime output seconds for '2012-12-24 16:30:00 +01:00
     ok 313 - strftime
-    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 316 - correct parse_datetime return value for '2012-12-24 14:30:00 -01:00'
     ok 317 - correct parse_datetime output seconds for '2012-12-24 14:30:00 -01:00
     ok 318 - strftime
-    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 321 - correct parse_datetime return value for '2012-12-24 16:30:00.123456 +01:00'
     ok 322 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456 +01:00
     ok 323 - strftime
-    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 326 - correct parse_datetime return value for '2012-12-24 14:30:00.123456 -01:00'
     ok 327 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456 -01:00
     ok 328 - strftime
-    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 331 - correct parse_datetime return value for '2012-12-24 15:30:00.123456 -00:00'
     ok 332 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456 -00:00
     ok 333 - strftime
-    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 336 - correct parse_datetime return value for '20121224T1630+01:00'
     ok 337 - correct parse_datetime output seconds for '20121224T1630+01:00
     ok 338 - strftime
-    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 341 - correct parse_datetime return value for '2012-12-24T1630+01:00'
     ok 342 - correct parse_datetime output seconds for '2012-12-24T1630+01:00
     ok 343 - strftime
-    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 346 - correct parse_datetime return value for '20121224T16:30+01'
     ok 347 - correct parse_datetime output seconds for '20121224T16:30+01
     ok 348 - strftime
-    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00
     ok 351 - correct parse_datetime return value for '20121224T16:30 +01'
     ok 352 - correct parse_datetime output seconds for '20121224T16:30 +01
     ok 353 - strftime
-    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
-    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
+    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00
 ok 1 - subtests
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [Tarantool-patches] [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (9 preceding siblings ...)
  2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 1/2] datetime: update tests for macosx Timur Safin via Tarantool-patches
@ 2021-08-03 21:23 ` Timur Safin via Tarantool-patches
  2021-08-06  0:30   ` Safin Timur via Tarantool-patches
  2021-08-03 21:26 ` [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
  11 siblings, 1 reply; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-03 21:23 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

To make Lua code less platfrom dependent we move
all platform specifics to the C level. Specifically
we want to avoid dependence on `struct tm {}` layout
and GLIBC `strftime` behaviour.
As a bonus we have that after moving of time dependent
code to C we make Lua code significantly simpler and thus
may get rid of all implementation specific details
with numerous typedefs and function declarations.

Part of #5941
---
 src/exports.h           |   4 ++
 src/lib/core/datetime.c |  63 +++++++++++++++++-
 src/lib/core/datetime.h |  23 ++++++-
 src/lua/datetime.lua    | 139 +++++-----------------------------------
 4 files changed, 104 insertions(+), 125 deletions(-)

diff --git a/src/exports.h b/src/exports.h
index 63efe0ec7..f4a6f98d7 100644
--- a/src/exports.h
+++ b/src/exports.h
@@ -218,6 +218,10 @@ EXPORT(curl_version)
 EXPORT(curl_version_info)
 #endif /* EXPORT_LIBCURL_SYMBOLS */
 EXPORT(datetime_pack)
+EXPORT(datetime_asctime)
+EXPORT(datetime_ctime)
+EXPORT(datetime_now)
+EXPORT(datetime_strftime)
 EXPORT(datetime_to_string)
 EXPORT(datetime_unpack)
 EXPORT(decimal_unpack)
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
index 03facb123..a921cbd11 100755
--- a/src/lib/core/datetime.c
+++ b/src/lib/core/datetime.c
@@ -30,6 +30,7 @@
  */
 
 #include <string.h>
+#include <time.h>
 
 #include "trivia/util.h"
 #include "datetime.h"
@@ -165,12 +166,35 @@ mp_encode_datetime(char *data, const struct datetime *date)
 	return datetime_pack(data, date);
 }
 
+static int
+local_dt(int64_t secs)
+{
+	return dt_from_rdn((int)(secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET);
+}
+
+static struct tm *
+datetime_to_tm(const struct datetime *date)
+{
+	static struct tm tm;
+
+	memset(&tm, 0, sizeof(tm));
+	int64_t secs = date->secs;
+	dt_to_struct_tm(local_dt(secs), &tm);
+
+	int seconds_of_day = date->secs % SECS_PER_DAY;
+	tm.tm_hour = (seconds_of_day / 3600) % 24;
+	tm.tm_min = (seconds_of_day / 60) % 60;
+	tm.tm_sec = seconds_of_day % 60;
+
+	return &tm;
+}
+
 int datetime_to_string(const struct datetime *date, char *buf, uint32_t len)
 {
 	char * src = buf;
 	int offset = date->offset;
 	int64_t secs = date->secs + offset * 60;
-	dt_t dt = dt_from_rdn((secs / SECS_PER_DAY) + 719163);
+	dt_t dt = local_dt(secs);
 
 	int year, month, day, sec, ns, sign;
 	dt_to_ymd(dt, &year, &month, &day);
@@ -214,6 +238,43 @@ int datetime_to_string(const struct datetime *date, char *buf, uint32_t len)
 	return (buf - src);
 }
 
+void
+datetime_now(struct datetime * now)
+{
+	struct timeval tv;
+	gettimeofday(&tv, NULL);
+	now->secs = tv.tv_sec;
+	now->nsec = tv.tv_usec * 1000;
+
+	time_t now_seconds;
+	time(&now_seconds);
+	struct tm tm;
+	localtime_r(&now_seconds, &tm);
+	now->offset = tm.tm_gmtoff / 60;
+}
+
+char *
+datetime_asctime(const struct datetime *date)
+{
+	struct tm *p_tm = datetime_to_tm(date);
+	return asctime(p_tm);
+}
+
+char *
+datetime_ctime(const struct datetime *date)
+{
+	time_t time = date->secs;
+	return ctime(&time);
+}
+
+size_t
+datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
+		  uint32_t len)
+{
+	struct tm *p_tm = datetime_to_tm(date);
+	return strftime(buf, len, fmt, p_tm);
+}
+
 int
 mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
 {
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
index 540bd68d9..012fb340d 100644
--- a/src/lib/core/datetime.h
+++ b/src/lib/core/datetime.h
@@ -42,6 +42,7 @@ extern "C"
 
 #ifndef SECS_PER_DAY
 #define SECS_PER_DAY	86400
+#define DT_EPOCH_1970_OFFSET 719163
 #endif
 
 /**
@@ -102,7 +103,27 @@ mp_encode_datetime(char *data, const struct datetime *date);
  * @param buf output character buffer
  * @param len size ofoutput buffer
  */
-int datetime_to_string(const struct datetime *date, char *buf, uint32_t len);
+int
+datetime_to_string(const struct datetime *date, char *buf, uint32_t len);
+
+/**
+ * Convert datetime to string using default asctime format
+ * "Sun Sep 16 01:03:52 1973\n\0"
+ * @param date source datetime value
+ * @sa datetime_ctime
+ */
+char *
+datetime_asctime(const struct datetime *date);
+
+char *
+datetime_ctime(const struct datetime *date);
+
+size_t
+datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
+		  uint32_t len);
+
+void
+datetime_now(struct datetime * now);
 
 int
 mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
index 5cad4e02f..1549735bd 100644
--- a/src/lua/datetime.lua
+++ b/src/lua/datetime.lua
@@ -82,90 +82,18 @@ ffi.cdef [[
     int
     datetime_to_string(const struct datetime * date, char *buf, uint32_t len);
 
+    char *
+    datetime_asctime(const struct datetime *date);
 
-    // <asm-generic/posix_types.h>
-    typedef long            __kernel_long_t;
-    typedef unsigned long   __kernel_ulong_t;
-    // /usr/include/x86_64-linux-gnu/bits/types/time_t.h
-    typedef long            time_t;
+    char *
+    datetime_ctime(const struct datetime *date);
 
+    size_t
+    datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
+                      uint32_t len);
 
-    // <time.h>
-    typedef __kernel_long_t	__kernel_time_t;
-    typedef __kernel_long_t	__kernel_suseconds_t;
-
-    struct timespec {
-        __kernel_time_t	        tv_sec;     /* seconds */
-        long                    tv_nsec;    /* nanoseconds */
-    };
-
-    struct timeval {
-        __kernel_time_t	        tv_sec;	    /* seconds */
-        __kernel_suseconds_t    tv_usec;    /* microseconds */
-    };
-
-    struct timezone {
-        int	tz_minuteswest;     /* minutes west of Greenwich */
-        int	tz_dsttime;	        /* type of dst correction */
-    };
-
-    // /usr/include/x86_64-linux-gnu/sys/time.h
-    typedef struct timezone * __timezone_ptr_t;
-
-    /* Get the current time of day and timezone information,
-       putting it into *TV and *TZ.  If TZ is NULL, *TZ is not filled.
-       Returns 0 on success, -1 on errors.
-
-       NOTE: This form of timezone information is obsolete.
-       Use the functions and variables declared in <time.h> instead.  */
-    int gettimeofday (struct timeval *__tv, struct timezone * __tz);
-
-    // /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h
-    /* ISO C `broken-down time' structure.  */
-    struct tm
-    {
-        int tm_sec;	        /* Seconds.	[0-60] (1 leap second) */
-        int tm_min;	        /* Minutes.	[0-59] */
-        int tm_hour;        /* Hours.	[0-23] */
-        int tm_mday;        /* Day.		[1-31] */
-        int tm_mon;	        /* Month.	[0-11] */
-        int tm_year;        /* Year	- 1900.  */
-        int tm_wday;        /* Day of week.	[0-6] */
-        int tm_yday;        /* Days in year.[0-365]	*/
-        int tm_isdst;       /* DST.		[-1/0/1]*/
-
-        long int tm_gmtoff; /* Seconds east of UTC.  */
-        const char *tm_zone;/* Timezone abbreviation.  */
-    };
-
-    // <time.h>
-    /* Return the current time and put it in *TIMER if TIMER is not NULL.  */
-    time_t time (time_t *__timer);
-
-    /* Format TP into S according to FORMAT.
-    Write no more than MAXSIZE characters and return the number
-    of characters written, or 0 if it would exceed MAXSIZE.  */
-    size_t strftime (char * __s, size_t __maxsize, const char * __format,
-                     const struct tm * __tp);
-
-    /* Parse S according to FORMAT and store binary time information in TP.
-    The return value is a pointer to the first unparsed character in S.  */
-    char *strptime (const char * __s, const char * __fmt, struct tm *__tp);
-
-    /* Return the `struct tm' representation of *TIMER in UTC,
-    using *TP to store the result.  */
-    struct tm *gmtime_r (const time_t * __timer, struct tm * __tp);
-
-    /* Return the `struct tm' representation of *TIMER in local time,
-    using *TP to store the result.  */
-    struct tm *localtime_r (const time_t * __timer, struct tm * __tp);
-
-    /* Return a string of the form "Day Mon dd hh:mm:ss yyyy\n"
-    that is the representation of TP in this format.  */
-    char *asctime (const struct tm *__tp);
-
-    /* Equivalent to `asctime (localtime (timer))'.  */
-    char *ctime (const time_t *__timer);
+    void
+    datetime_now(struct datetime * now);
 
 ]]
 
@@ -921,62 +849,27 @@ local function datetime_from(o)
 end
 
 local function local_now()
-    local p_tv = ffi.new('struct timeval [1]')
-    local rc = builtin.gettimeofday(p_tv, nil)
-    assert(rc == 0)
-
-    local secs = p_tv[0].tv_sec
-    local nsec = p_tv[0].tv_usec * 1000
-
-    local p_time = ffi.new('time_t[1]')
-    local p_tm = ffi.new('struct tm[1]')
-    builtin.time(p_time)
-    builtin.localtime_r(p_time, p_tm)
-    local ofs = p_tm[0].tm_gmtoff / 60 -- convert seconds to minutes
-
-    return datetime_new_raw(secs, nsec, ofs)
-end
-
-local function datetime_to_tm_ptr(o)
-    assert(is_datetime(o))
-    local p_tm = ffi.new('struct tm[1]')
-    -- dt_to_struct_tm() fills only date data
-    builtin.dt_to_struct_tm(local_dt(o), p_tm)
-
-    -- calculate the smaller data (hour, minute,
-    -- seconds) using datetime seconds value
-    local seconds_of_day = o.secs % 86400
-    local hour = (seconds_of_day / 3600) % 24
-    local minute = (seconds_of_day / 60) % 60
-    p_tm[0].tm_sec = seconds_of_day % 60
-    p_tm[0].tm_min = minute
-    p_tm[0].tm_hour = hour
-
-    p_tm[0].tm_gmtoff = o.offset * 60
-
-    return p_tm
+    local d = datetime_new_raw(0, 0, 0)
+    builtin.datetime_now(d)
+    return d
 end
 
 local function asctime(o)
     check_date(o, "datetime:asctime()")
 
-    local p_tm = datetime_to_tm_ptr(o)
-    return ffi.string(builtin.asctime(p_tm))
+    return ffi.string(builtin.datetime_asctime(o))
 end
 
 local function ctime(o)
     check_date(o, "datetime:ctime()")
-    local p_time = ffi.new('time_t[1]')
-    p_time[0] = o.secs
-    return ffi.string(builtin.ctime(p_time))
+    return ffi.string(builtin.datetime_ctime(o))
 end
 
 local function strftime(fmt, o)
     check_date(o, "datetime.strftime()")
-    local p_tm = datetime_to_tm_ptr(o)
-    local sz = builtin.strftime(nil, 1024, fmt, p_tm) + 1
+    local sz = 128
     local buff = ffi.new('char[?]', sz)
-    builtin.strftime(buff, sz, fmt, p_tm)
+    builtin.datetime_strftime(o, fmt, buff, sz)
     return ffi.string(buff)
 end
 
-- 
2.29.2


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 0/9] Initial datetime support
  2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
                   ` (10 preceding siblings ...)
  2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers Timur Safin via Tarantool-patches
@ 2021-08-03 21:26 ` Timur Safin via Tarantool-patches
  11 siblings, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-03 21:26 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

I've sent a couple of extra patches, which became part of 
This patch-set:

- [PATCH v3 1/2] datetime: update tests for macosx
- [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers

Those 2 patches fixed CI for FreeBSD and MacOSX.

Timur

> -----Original Message-----
> From: Timur Safin <tsafin@tarantool.org>
> Sent: Monday, August 2, 2021 3:41 AM
> To: v.shpilevoy@tarantool.org; olegrok@tarantool.org
> Cc: Timur Safin <tsafin@tarantool.org>; imun@tarantool.org;
> imeevma@tarantool.org; tarantool-patches@dev.tarantool.org
> Subject: [PATCH v3 0/9] Initial datetime support
> 
> * Version #3 changes:
> 
>   - renamed `struct datetime_t` to `struct datetime`, and `struct
>     datetime_interval_t` to `struct datetime_interval`;
>   - significantly reworked arguments checks in module api entries;
>   - fixed datetime comparisons;
>   - changed hints calculation to take into account fractional part;
>   - provided more comments here and there;
> 
> NB! There are MacOSX problems due to GLIBC specific code used (as Vlad
>     has already pointed out) - so additional patch, making it more
>     cross-compatible coming here soon...
> 
> * Version #2 changes:
> 
>   - fixed problem with overloaded '-' and '+' operations for datetime
>     arguments;
>   - fixed messagepack serialization problems;
>   - heavily documented MessagePack serialization schema in the code;
>   - introduced working implementation of datetime hints for storage engines;
>   - made interval related names be more consistent, renamed durations and
> period
>     to intervals, i.e. t_datetime_duration to datetime_interval_t,
>     duration_* to interval_*, period to interval;
>   - properly implemented all reasonable cases of datetime+interval
>     arithmetic;
>   - moved all initialization code to utils.c;
>   - renamed core/mp_datetime.c to core/datetime.c because it makes more
>     sense now;
> 
> * Version #1 - initial RFC series
> 
> In brief
> --------
> This patchset implements datetime lua support in box, with serialization
> to messagepack, yaml, json and lua mode. Also it contains storage
> engines' indices implementation for datetime type introduced.
> 
> * Current implementation is heavily influenced by Sci-Lua lua-time module
>   https://github.com/stepelu/lua-time
>   e.g. you could find very similar approach for handling of operations
>   with year or month long intervals (which should be handled differently
> than
>   usual intervals of seconds, or days).
> 
> * But internally we actually use Christian Hanson' c-dt module
>   https://github.com/chansen/c-dt
>   (though it has been modified slightly for cleaner integration
>   into cmake build process)
> 
> 
> Datetime Module API
> -------------------
> 
> We used to have here draft documentation of datetime module api, but
> for a convenience it has been extracted to the discussion topic there -
> https://github.com/tarantool/tarantool/discussions/6244#discussioncomment-
> 1043988
> 
> Messagepack serialization schema
> --------------------------------
> 
> In short it looks like:
> - now we introduce new MP_EXT extension type #4;
> - we may save 1 required and 2 optional fields for datetime field;
> 
> In all gory details it's explained in MessagePack serialization schema
> depicted here:
> https://github.com/tarantool/tarantool/discussions/6244#discussioncomment-
> 1043990
> 
> 
> https://github.com/tarantool/tarantool/issues/5941
> https://github.com/tarantool/tarantool/issues/5946
> 
> https://github.com/tarantool/tarantool/tree/tsafin/gh-5941-datetime-v3
> 
> Timur Safin (9):
>   build: add Christian Hansen c-dt to the build
>   lua: built-in module datetime
>   lua, datetime: datetime tests
>   lua, datetime: display datetime
>   box, datetime: messagepack support for datetime
>   box, datetime: datetime comparison for indices
>   lua, datetime: time intervals support
>   datetime: changelog for datetime module
>   lua, box, datetime: rename struct datetime_t
> 
>  .gitmodules                                   |    3 +
>  CMakeLists.txt                                |    8 +
>  .../gh-5941-datetime-type-support.md          |    4 +
>  cmake/BuildCDT.cmake                          |    6 +
>  src/CMakeLists.txt                            |    3 +
>  src/box/field_def.c                           |   52 +-
>  src/box/field_def.h                           |    4 +
>  src/box/lua/serialize_lua.c                   |    7 +-
>  src/box/memtx_space.c                         |    3 +-
>  src/box/msgpack.c                             |    7 +-
>  src/box/tuple_compare.cc                      |   76 ++
>  src/box/vinyl.c                               |    3 +-
>  src/exports.h                                 |   29 +
>  src/lib/core/CMakeLists.txt                   |    4 +-
>  src/lib/core/datetime.c                       |  250 ++++
>  src/lib/core/datetime.h                       |  115 ++
>  src/lib/core/mp_extension_types.h             |    1 +
>  src/lib/mpstream/mpstream.c                   |   10 +
>  src/lib/mpstream/mpstream.h                   |    3 +
>  src/lua/datetime.lua                          | 1058 +++++++++++++++++
>  src/lua/init.c                                |    4 +-
>  src/lua/msgpack.c                             |   12 +
>  src/lua/msgpackffi.lua                        |   18 +
>  src/lua/serializer.c                          |    4 +
>  src/lua/serializer.h                          |    2 +
>  src/lua/utils.c                               |   28 +-
>  src/lua/utils.h                               |   12 +
>  test/app-tap/datetime.test.lua                |  367 ++++++
>  test/engine/datetime.result                   |   77 ++
>  test/engine/datetime.test.lua                 |   35 +
>  test/unit/CMakeLists.txt                      |    2 +
>  test/unit/datetime.c                          |  221 ++++
>  test/unit/datetime.result                     |  358 ++++++
>  third_party/c-dt                              |    1 +
>  third_party/lua-cjson/lua_cjson.c             |    8 +
>  third_party/lua-yaml/lyaml.cc                 |    6 +-
>  36 files changed, 2776 insertions(+), 25 deletions(-)
>  create mode 100644 changelogs/unreleased/gh-5941-datetime-type-support.md
>  create mode 100644 cmake/BuildCDT.cmake
>  create mode 100755 src/lib/core/datetime.c
>  create mode 100644 src/lib/core/datetime.h
>  create mode 100644 src/lua/datetime.lua
>  create mode 100755 test/app-tap/datetime.test.lua
>  create mode 100644 test/engine/datetime.result
>  create mode 100644 test/engine/datetime.test.lua
>  create mode 100644 test/unit/datetime.c
>  create mode 100644 test/unit/datetime.result
>  create mode 160000 third_party/c-dt
> 
> --
> 2.29.2



^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices
  2021-08-03 12:02   ` Serge Petrenko via Tarantool-patches
  2021-08-03 12:59     ` Timur Safin via Tarantool-patches
@ 2021-08-04 10:12     ` Timur Safin via Tarantool-patches
  1 sibling, 0 replies; 38+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-04 10:12 UTC (permalink / raw)
  To: 'Serge Petrenko', v.shpilevoy, olegrok; +Cc: tarantool-patches

Hello Sergey,

Here is small update since previous message...

> > You just need to assign the same HINT_VALUE_MAX hint to every
> datetime value
> > with date->secs >= HINT_VALUE_SECS_MAX. Otherwise the comparison
> would
> > make mistakes (judging by hint values only) for such far away
> dates.
> 
> Yes, thanks for the note, will make sure that values which exceed
> selected
> 36-bit will stick to the maximum value.

Introduced this simple fix to the current code:
----------------------------------------
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index f733b9f01..0c5e0b666 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -1659,7 +1659,8 @@ hint_datetime(struct datetime_t *date)
        int32_t nsec = date->nsec;
        uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 :
                        secs - HINT_VALUE_SECS_MIN;
-       val &= HINT_VALUE_SECS_MAX;
+       if (val >= HINT_VALUE_SECS_MAX)
+               val = HINT_VALUE_SECS_MAX;
        val <<= HINT_VALUE_NSEC_BITS;
        val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX;
        return hint_create(MP_CLASS_DATETIME, val);
----------------------------------------

Updated accordingly and force-pushed to the branch. 

> > But you don't allow datetime in SCALAR fields, as I see in the
> previous
> > commit.
> > So you don't need to account for datetime in scalar hints.
> 
> That's bug - every type is scalar type, unless it's collection (map
> or array)
> so datetime should be considered as scalar. That was oversight if
> not.
> Thanks!

Also DATETIME is considered compatible with SCALAR type in 
MessagePack and with scalar column. Also updated patch, and force-pushed
to branch.

Thanks,
Timur


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build Timur Safin via Tarantool-patches
@ 2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-05  8:55     ` Safin Timur via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-04 23:58 UTC (permalink / raw)
  To: Timur Safin, olegrok; +Cc: tarantool-patches

Hi! Thanks for the patch!

On 02.08.2021 02:40, Timur Safin via Tarantool-patches wrote:
> * Integrated chansen/c-dt parser as 3rd party module to the
>   Tarantool cmake build process.
> * Points to tsafin/c-dt instead iof original chansen/c-dt to
>   have easier build integration, because there is additional
>   commit which integrated cmake support

As I said in the previous review, it points at
https://github.com/tarantool/c-dt.git, not at tsafin/c-dt. Why
do you keep saying the contrary? Look:

	url = https://github.com/tarantool/c-dt.git

It is 'tarantool', not 'tsafin' here.

Also, like on the previous review, when I call 'make' and do
'git status', I see:

	Changes not staged for commit:
	  (use "git add <file>..." to update what will be committed)
	  (use "git restore <file>..." to discard changes in working directory)
	  (commit or discard the untracked or modified content in submodules)
		modified:   third_party/c-dt (modified content, untracked content)

You still didn't fix it. Please, do.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime Timur Safin via Tarantool-patches
@ 2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-06  0:23     ` Safin Timur via Tarantool-patches
  2021-08-06  0:26   ` Safin Timur via Tarantool-patches
  2021-08-08 14:34   ` Vladislav Shpilevoy via Tarantool-patches
  2 siblings, 1 reply; 38+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-04 23:58 UTC (permalink / raw)
  To: Timur Safin, olegrok; +Cc: tarantool-patches

Thanks for the patch!

See 28 comments below.

I am asking you not to rush answering 'will fix', 'ok',
'good idea' etc. It seems you are missing a lot of comments,
probably because of that. You need to answer only when you
already did the changes, preferably with the diff showing
the change in the corresponding commit under each comment. Or
when you have questions. While doing review fixed you need
to keep track of the comments so as not to loose any of them.

> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> new file mode 100644
> index 000000000..84d6642ab
> --- /dev/null
> +++ b/src/lib/core/datetime.h
> @@ -0,0 +1,112 @@
> +#pragma once
> +/*
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * 1. Redistributions of source code must retain the above
> + *    copyright notice, this list of conditions and the
> + *    following disclaimer.
> + *
> + * 2. Redistributions in binary form must reproduce the above
> + *    copyright notice, this list of conditions and the following
> + *    disclaimer in the documentation and/or other materials
> + *    provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
> + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
> + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
> + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
> + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include "c-dt/dt.h"
> +
> +#if defined(__cplusplus)
> +extern "C"
> +{
> +#endif /* defined(__cplusplus) */
> +
> +/**
> + * Full datetime structure representing moments
> + * since Unix Epoch (1970-01-01).
> + * Time is kept normalized to UTC, time-zone offset
> + * is informative only.
> + */
> +struct datetime_t {

1. Well. How can we proceed beyond the second commit, if you keep
ignoring my comments? Please, drop _t suffix. Please, comply with
our code style for comments (they should not be on the same line as
the members, and should not use any signs like '<').

If you did it in some later commits - it is not atomic anymore. The
change must be self-sufficient. This commit is not. It depends on
one of the last commits which renames something. Why couldn't you
give the proper names from the beginning, from this commit?

And what is worse - this entire file is not needed in this commit,
I already said it before. It is simply not used here at all. Please,
move it to the relevant commit.

What is even further worse - you added here some functions which
do not have any definition. You only added .h file. All the functions
here are simply undefined. Wtf?

> +	int64_t secs;	/**< seconds since epoch */
> +	int32_t nsec;	/**< nanoseconds if any */
> +	int32_t offset; /**< offset in minutes from UTC */
> +};
> +
> +/**
> + * Date/time interval structure
> + */
> +struct datetime_interval_t {
> +	int64_t secs; /**< relative seconds delta */
> +	int32_t nsec; /**< nanoseconds delta */
> +};
> +
> +int
> +datetime_compare(const struct datetime_t * lhs,
> +		 const struct datetime_t * rhs);
> +
> +
> +struct datetime_t *
> +datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
> +
> +/**
> + * Pack datetime_t data to the MessagePack buffer.
> + */
> +char *
> +datetime_pack(char *data, const struct datetime_t *date);
> +
> +/**
> + * Calculate size of MessagePack buffer for datetime_t data.
> + */
> +uint32_t
> +mp_sizeof_datetime(const struct datetime_t *date);
> +
> +/**
> + * Decode data from MessagePack buffer to datetime_t structure.
> + */
> +struct datetime_t *
> +mp_decode_datetime(const char **data, struct datetime_t *date);
> +
> +/**
> + * Encode datetime_t structure to the MessagePack buffer.
> + */
> +char *
> +mp_encode_datetime(char *data, const struct datetime_t *date);
> +
> +/**
> + * Convert datetime to string using default format
> + * @param date source datetime value
> + * @param buf output character buffer
> + * @param len size ofoutput buffer
> + */
> +int
> +datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
> +
> +int
> +mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
> +
> +int
> +mp_fprint_datetime(FILE *file, const char **data, uint32_t len);

2. Please, try to follow decimal and uuid here. They have separate files
for working with MessagePack: tt_uuid.h/mp_uuid.h, decimal.h/mp_decimal.h.
You should have datetime.h and mp_datetime.h.

Also once you add these to_string, mp_fprint, mp_snprint, pack/unpack,
you need to add unitests in C for all these functions. See how UUID does
it in test/unit/uuid.c and decimal in test/unit/decimal.c.

> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> new file mode 100644
> index 000000000..870edc541
> --- /dev/null
> +++ b/src/lua/datetime.lua

<...>

> +    // /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h
> +    /* ISO C `broken-down time' structure.  */
> +    struct tm
> +    {
> +        int tm_sec;	        /* Seconds.	[0-60] (1 leap second) */
> +        int tm_min;	        /* Minutes.	[0-59] */
> +        int tm_hour;        /* Hours.	[0-23] */
> +        int tm_mday;        /* Day.		[1-31] */
> +        int tm_mon;	        /* Month.	[0-11] */
> +        int tm_year;        /* Year	- 1900.  */
> +        int tm_wday;        /* Day of week.	[0-6] */
> +        int tm_yday;        /* Days in year.[0-365]	*/
> +        int tm_isdst;       /* DST.		[-1/0/1]*/
> +
> +        long int tm_gmtoff; /* Seconds east of UTC.  */
> +        const char *tm_zone;/* Timezone abbreviation.  */

3. Didn't we already discuss it? The structure is platform-specific.
You can't use it safely via FFI. And you even agreed with that, but
still the struct is here. Why?

> +    };

<...>

> +local datetime_t = ffi.typeof('struct datetime_t')
> +local interval_t = ffi.typeof('struct datetime_interval_t')
> +
> +local function is_interval(o)
> +    return type(o) == 'cdata' and ffi.istype(interval_t, o)
> +end
> +
> +local function is_datetime(o)
> +    return type(o) == 'cdata' and ffi.istype(datetime_t, o)
> +end
> +
> +local function is_date_interval(o)
> +    return is_datetime(o) or is_interval(o)

4. You make type(o) == 'cdata' check 2 times here.

> +end
> +
> +local function interval_new()
> +    local interval = ffi.new(interval_t)
> +    return interval
> +end
> +
> +local function check_date(o, message, lvl)

5. 'lvl' is never used by any of the invocations. You waste time on
the 'if', so it can be dropped. Or start using 'lvl' but never pass it
as nil, please. The same for all the other functions below taking
'lvl'.

> +    if lvl == nil then
> +        lvl = 2
> +    end
> +    if not is_datetime(o) then
> +        return error(("%s: expected datetime, but received %s"):
> +                     format(message, o), lvl)
> +    end
> +end
<...>

> +
> +local function datetime_cmp(lhs, rhs)

6. This does not look correct:

tarantool> datetime = require('datetime')
---
...

tarantool> d = datetime()
---
...

tarantool> d < 'bad string'
---
- false
...

tarantool> d > 'bad string'
---
- false
...

Do you have any tests? In this commit I see zero tests for
your code. Only very few tests for dt library. Please, add
tests for everything you have here.

> +    if not is_date_interval(lhs) or
> +       not is_date_interval(rhs) then
> +       return nil
> +    end
> +    local sdiff = lhs.secs - rhs.secs
> +    return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
> +end
> +
> +local function datetime_eq(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc == 0 or false
> +end
> +
> +local function datetime_lt(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc < 0 or false
> +end
> +
> +local function datetime_le(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc <= 0 or false
> +end
> +
> +local function datetime_serialize(self)
> +    -- Allow YAML, MsgPack and JSON to dump objects with sockets
> +    return { secs = self.secs, nsec = self.nsec, offset = self.offset }
> +end
> +
> +local function interval_serialize(self)
> +    -- Allow YAML and JSON to dump objects with sockets

7. What do these comments mean? Why is it specific to these
formats? And why doesn this comment include MsgPack while the
previous one does?

> +    return { secs = self.secs, nsec = self.nsec }
> +end
> +
> +local function local_rd(o)
> +    return math.floor(tonumber(o.secs / SECS_PER_DAY)) + DT_EPOCH_1970_OFFSET

8. I suggest to cache math.floor into a global variable. The same
for the other math.* usages in this file. To save one index operation.

> +end
> +
> +local function local_dt(o)
> +    return builtin.dt_from_rdn(local_rd(o))
> +end
> +
> +local function _normalize_nsec(secs, nsec)

9. Why do you have '_' prefix? The function is not global, you
don't need any protection against name conflicts.

> +    if nsec < 0 then
> +        secs = secs - 1
> +        nsec = nsec + NANOS_PER_SEC
> +    elseif nsec >= NANOS_PER_SEC then
> +        secs = secs + 1
> +        nsec = nsec - NANOS_PER_SEC
> +    end
> +    return secs, nsec
> +end
> +
> +local function check_range(v, range, txt)
> +    assert(#range == 2)
> +    if not (v >= range[1] and v <= range[2]) then

10. Wouldn't it be shorter to make

	if v < range[1] or v > range[2] then

?

> +        error(('value %d of %s is out of allowed range [%d, %d]'):
> +              format(v, txt, range[1], range[2]))
> +    end
> +end
> +
> +local datetime_index_handlers = {
> +    unixtime = function(self)
> +        return self.secs
> +    end,
> +
> +    timestamp = function(self)
> +        return tonumber(self.secs) + self.nsec / 1e9

11. If you are saying the Lua number value range is enough,
why do you store them as 64bit integers in 'self' and
convert to a number only for the external API?

> +    end,
> +
> +    nanoseconds = function(self)
> +        return self.secs * 1e9 + self.nsec
> +    end,
> +
> +    microseconds = function(self)
> +        return self.secs * 1e6 + self.nsec / 1e3
> +    end,
> +
> +    milliseconds = function(self)
> +        return self.secs * 1e3 + self.nsec / 1e6
> +    end,
> +
> +    seconds = function(self)
> +        return tonumber(self.secs) + self.nsec / 1e9
> +    end,
> +
> +    minutes = function(self)
> +        return (tonumber(self.secs) + self.nsec / 1e9) / 60
> +    end,
> +
> +    hours = function(self)
> +        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60)
> +    end,
> +
> +    days = function(self)
> +        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60) / 24

12. Is Lua parser able to merge the last constants into one
number to make only one division at runtime? If not, I suggest
to do it manually.

> +    end,
> +}
> +
> +local datetime_index = function(self, key)
> +    return datetime_index_handlers[key] ~= nil and
> +           datetime_index_handlers[key](self) or nil

13. To save one hash lookup operation you can do this:

	local h = datetime_index_handlers[key]
	return h and h(self)

The same for datetime_newindex.

> +end
> +
> +local datetime_newindex_handlers = {
> +    unixtime = function(self, value)
> +        self.secs = value
> +        self.nsec, self.offset = 0, 0
> +    end,
> +
> +    timestamp = function(self, value)
> +        local secs, frac = math.modf(value)
> +        self.secs = secs
> +        self.nsec = frac * 1e9
> +        self.offset = 0
> +    end,
> +}
> +
> +local function datetime_newindex(self, key, value)
> +    if datetime_newindex_handlers[key] ~= nil then
> +        datetime_newindex_handlers[key](self, value)
> +    end
> +end
> +
> +local function datetime_new_raw(secs, nsec, offset)
> +    local dt_obj = ffi.new(datetime_t)
> +    dt_obj.secs = secs
> +    dt_obj.nsec = nsec
> +    dt_obj.offset = offset
> +    return dt_obj
> +end
> +
> +local function mk_timestamp(dt, sp, fp, offset)

14. What are 'sp' and 'fp'?

15. The function is a datetime constructor, because it creates and
returns a new datetime object. Please, call it correspondingly
then. For instance, `datetime_new_dt()`.

> +    local epochV = dt ~= nil and (builtin.dt_rdn(dt) - DT_EPOCH_1970_OFFSET) *
> +                   SECS_PER_DAY or 0
> +    local spV = sp ~= nil and sp or 0
> +    local fpV = fp ~= nil and fp or 0
> +    local ofsV = offset ~= nil and offset or 0
> +    return datetime_new_raw (epochV + spV - ofsV * 60, fpV, ofsV)
> +end
> +
> +-- create @datetime_t given object @o fields

16. Didn't we agree in the previous review that you will stop using
name 'o'? At least in the context where there are zeros. It is very
confusing.

> +local function datetime_new(o)
> +    if o == nil then
> +        return datetime_new_raw(0, 0, 0)
> +    end
> +    local secs = 0
> +    local nsec = 0
> +    local offset = 0
> +    local easy_way = false
> +    local y = 0
> +    local M = 0
> +    local d = 0
> +    local ymd = false
> +
> +    local h = 0
> +    local m = 0
> +    local s = 0
> +    local frac = 0
> +    local hms = false
> +
> +    local dt = 0
> +
> +    local handlers = {

17. Please, stop using closures in the constructor of a
potentionally very frequently created/deleted object.

> +        secs = function(_, v)
> +            secs = v
> +            easy_way = true
> +        end,
> +
> +        nsec = function(_, v)
> +            nsec = v
> +            easy_way = true
> +        end,
> +
> +        offset = function (_, v)
> +            offset = v
> +            easy_way = true
> +        end,
> +
> +        year = function(k, v)
> +            check_range(v, {1, 9999}, k)
> +            y = v
> +            ymd = true
> +        end,
> +
> +        month = function(k, v)
> +            check_range(v, {1, 12}, k)
> +            M = v
> +            ymd = true
> +        end,
> +
> +        day = function(k, v)
> +            check_range(v, {1, 31}, k)
> +            d = v
> +            ymd = true
> +        end,
> +
> +        hour = function(k, v)
> +            check_range(v, {0, 23}, k)
> +            h = v
> +            hms = true
> +        end,
> +
> +        minute = function(k, v)
> +            check_range(v, {0, 59}, k)
> +            m = v
> +            hms = true
> +        end,
> +
> +        second = function(k, v)
> +            check_range(v, {0, 60}, k)
> +            s, frac = math.modf(v)
> +            frac = frac * 1e9 -- convert fraction to nanoseconds
> +            hms = true
> +        end,
> +
> +        -- tz offset in minutes
> +        tz = function(k, v)
> +            check_range(v, {0, 720}, k)
> +            offset = v
> +        end
> +    }
> +    for key, value in pairs(o) do
> +        handlers[key](key, value)
> +    end
> +
> +    -- .sec, .nsec, .offset
> +    if easy_way then
> +        return datetime_new_raw(secs, nsec, offset)
> +    end
> +
> +    -- .year, .month, .day
> +    if ymd then
> +        dt = dt + builtin.dt_from_ymd(y, M, d)
> +    end
> +
> +    -- .hour, .minute, .second
> +    if hms then
> +        secs = h * 3600 + m * 60 + s
> +    end
> +
> +    return mk_timestamp(dt, secs, frac, offset)
> +end

18. It fails with a bad error:

tarantool> d = datetime({garbage = true})
---
- error: 'builtin/datetime.lua:449: attempt to call a nil value'
...


19. It can ignore some fields (here it ignores 'year'):

tarantool> d1 = datetime({secs = 100, year = 1})
---
...

tarantool> d2 = datetime({secs = 100, year = 2})
---
...

tarantool> d1 == d2
---
- true
...


20. Serialization does not seem to be working in the console
despite you tried to implement it above:

tarantool> d = datetime({secs = 100})
---
...

tarantool> d
---
- 'cdata<struct datetime_t>: 0x0103ad9c50'
...

> +
> +local function date_first(lhs, rhs)
> +    if is_datetime(lhs) then
> +        return lhs, rhs
> +    else
> +        return rhs, lhs
> +    end
> +end
> +
> +local function error_incompatible(name)
> +    error(("datetime:%s() - incompatible type of arguments"):
> +          format(name), 3)
> +end
> +
> +local function datetime_sub(lhs, rhs)
> +    check_date_interval(lhs, "operator -")
> +    local d, s = lhs, rhs
> +    local left_t = ffi.typeof(d)
> +    local right_t = ffi.typeof(s)
> +    local o
> +
> +    if left_t == datetime_t then
> +        -- left is date, right is date or generic interval
> +        if (right_t == datetime_t or right_t == interval_t) then

21. You do not need () around the condition in Lua.

> +            o = right_t == datetime_t and interval_new() or datetime_new()
> +            o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
> +                                            lhs.nsec - rhs.nsec)
> +            return o
> +        else
> +            error_incompatible("operator -")
> +        end
> +    -- both left and right are generic intervals
> +    elseif left_t == interval_t and right_t == interval_t then
> +        o = interval_new()
> +        o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
> +                                        lhs.nsec - rhs.nsec)
> +        return o
> +    else
> +        error_incompatible("operator -")
> +    end
> +end

22. Did you really test this function?

tarantool> dt = require('datetime')
---
...

tarantool> d1 = dt()
---
...

tarantool> d2 - 1
---
- error: 'builtin/datetime.lua:487: bad argument #1 to ''typeof'' (C type expected,
    got number)'
...

> +
> +local function datetime_add(lhs, rhs)
> +    local d, s = date_first(lhs, rhs)
> +
> +    check_date_interval(d, "operator +")
> +    check_interval(s, "operator +")
> +    local left_t = ffi.typeof(d)
> +    local right_t = ffi.typeof(s)
> +    local o
> +
> +    -- left is date, right is date or interval
> +    if left_t == datetime_t and right_t == interval_t then
> +        o = datetime_new()
> +        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
> +        return o
> +    -- both left and right are generic intervals
> +    elseif left_t == interval_t and right_t == interval_t then
> +        o = interval_new()
> +        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
> +        return o
> +    else
> +        error_incompatible("operator +")
> +    end
> +end
> +
> +-- simple parse functions:
> +-- parse_date/parse_time/parse_zone

23. As I said in the previous review, the comment is useless.
You just listed 3 function names following the comment. What
is the purpose of that? What should I learn from it? Please,
drop it.

<...>

> +
> +local function datetime_from(o)
> +    if o == nil or type(o) == 'table' then
> +        return datetime_new(o)
> +    elseif type(o) == 'string' then
> +        return parse(o)
> +    end
> +end
> +
> +local function local_now()
> +    local p_tv = ffi.new('struct timeval [1]')
> +    local rc = builtin.gettimeofday(p_tv, nil)

24. Why can't you use fiber time? It should be enough in
nearly all usecases and is much faster. For real-real time
talking directly to the kernel we have 'clock' Lua module
already. You could check if results of clock functions can
be passed into your module to create 'real' timestamps not
depending on the eventloop.

> +    assert(rc == 0)
> +
> +    local secs = p_tv[0].tv_sec
> +    local nsec = p_tv[0].tv_usec * 1000
> +
> +    local p_time = ffi.new('time_t[1]')
> +    local p_tm = ffi.new('struct tm[1]')
> +    builtin.time(p_time)
> +    builtin.localtime_r(p_time, p_tm)
> +    local ofs = p_tm[0].tm_gmtoff / 60 -- convert seconds to minutes
> +
> +    return datetime_new_raw(secs, nsec, ofs)
> +end
> +
> +local function datetime_to_tm_ptr(o)
> +    assert(is_datetime(o))
> +    local p_tm = ffi.new('struct tm[1]')
> +    -- dt_to_struct_tm() fills only date data
> +    builtin.dt_to_struct_tm(local_dt(o), p_tm)
> +
> +    -- calculate the smaller data (hour, minute,
> +    -- seconds) using datetime seconds value
> +    local seconds_of_day = o.secs % 86400
> +    local hour = (seconds_of_day / 3600) % 24
> +    local minute = (seconds_of_day / 60) % 60
> +    p_tm[0].tm_sec = seconds_of_day % 60
> +    p_tm[0].tm_min = minute
> +    p_tm[0].tm_hour = hour
> +
> +    p_tm[0].tm_gmtoff = o.offset * 60
> +
> +    return p_tm
> +end
> +
> +local function asctime(o)
> +    check_date(o, "datetime:asctime()")
> +
> +    local p_tm = datetime_to_tm_ptr(o)
> +    return ffi.string(builtin.asctime(p_tm))
> +end
> +
> +local function ctime(o)
> +    check_date(o, "datetime:ctime()")
> +    local p_time = ffi.new('time_t[1]')
> +    p_time[0] = o.secs
> +    return ffi.string(builtin.ctime(p_time))

25. Functions like ctime can't be used in Lua, because according
to their Linux man:

	The return value points to a statically allocated string
	which might be overwritten by subsequent calls to any of the
	date and time functions

Which means they use a global buffer. If you will
use datetime module in any of __gc handlers, it will
return garbage sometimes or just override results
somethere. That is exactly the same problem as existed
with tarantool's static buffer.

> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
> new file mode 100644
> index 000000000..b6f568c03
> --- /dev/null
> +++ b/test/unit/datetime.c

26. This file tests dt library. It does not test your code. Please, add
tests for everything you implemented in this commit. For c-dt library
the tests must be added in the previous commit, and the test file
must be called test/unit/cdt.c then.

> @@ -0,0 +1,221 @@
> +#include "dt.h"
> +#include <assert.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <time.h>
> +
> +#include "unit.h"
> +
> +const char sample[] = "2012-12-24T15:30Z";

27. Please, make it static.

> +
> +#define S(s) {s, sizeof(s) - 1}
> +struct {
> +	const char * sz;
> +	size_t len;
> +} tests[] = {
> +	S("2012-12-24 15:30Z"),
> +	S("2012-12-24 15:30z"),
> +	S("2012-12-24 15:30"),
> +	S("2012-12-24 16:30+01:00"),
> +	S("2012-12-24 16:30+0100"),
> +	S("2012-12-24 16:30+01"),
> +	S("2012-12-24 14:30-01:00"),
> +	S("2012-12-24 14:30-0100"),
> +	S("2012-12-24 14:30-01"),
> +	S("2012-12-24 15:30:00Z"),
> +	S("2012-12-24 15:30:00z"),
> +	S("2012-12-24 15:30:00"),
> +	S("2012-12-24 16:30:00+01:00"),
> +	S("2012-12-24 16:30:00+0100"),
> +	S("2012-12-24 14:30:00-01:00"),
> +	S("2012-12-24 14:30:00-0100"),
> +	S("2012-12-24 15:30:00.123456Z"),
> +	S("2012-12-24 15:30:00.123456z"),
> +	S("2012-12-24 15:30:00.123456"),
> +	S("2012-12-24 16:30:00.123456+01:00"),
> +	S("2012-12-24 16:30:00.123456+01"),
> +	S("2012-12-24 14:30:00.123456-01:00"),
> +	S("2012-12-24 14:30:00.123456-01"),
> +	S("2012-12-24t15:30Z"),
> +	S("2012-12-24t15:30z"),
> +	S("2012-12-24t15:30"),
> +	S("2012-12-24t16:30+01:00"),
> +	S("2012-12-24t16:30+0100"),
> +	S("2012-12-24t14:30-01:00"),
> +	S("2012-12-24t14:30-0100"),
> +	S("2012-12-24t15:30:00Z"),
> +	S("2012-12-24t15:30:00z"),
> +	S("2012-12-24t15:30:00"),
> +	S("2012-12-24t16:30:00+01:00"),
> +	S("2012-12-24t16:30:00+0100"),
> +	S("2012-12-24t14:30:00-01:00"),
> +	S("2012-12-24t14:30:00-0100"),
> +	S("2012-12-24t15:30:00.123456Z"),
> +	S("2012-12-24t15:30:00.123456z"),
> +	S("2012-12-24t16:30:00.123456+01:00"),
> +	S("2012-12-24t14:30:00.123456-01:00"),
> +	S("2012-12-24 16:30 +01:00"),
> +	S("2012-12-24 14:30 -01:00"),
> +	S("2012-12-24 15:30 UTC"),
> +	S("2012-12-24 16:30 UTC+1"),
> +	S("2012-12-24 16:30 UTC+01"),
> +	S("2012-12-24 16:30 UTC+0100"),
> +	S("2012-12-24 16:30 UTC+01:00"),
> +	S("2012-12-24 14:30 UTC-1"),
> +	S("2012-12-24 14:30 UTC-01"),
> +	S("2012-12-24 14:30 UTC-01:00"),
> +	S("2012-12-24 14:30 UTC-0100"),
> +	S("2012-12-24 15:30 GMT"),
> +	S("2012-12-24 16:30 GMT+1"),
> +	S("2012-12-24 16:30 GMT+01"),
> +	S("2012-12-24 16:30 GMT+0100"),
> +	S("2012-12-24 16:30 GMT+01:00"),
> +	S("2012-12-24 14:30 GMT-1"),
> +	S("2012-12-24 14:30 GMT-01"),
> +	S("2012-12-24 14:30 GMT-01:00"),
> +	S("2012-12-24 14:30 GMT-0100"),
> +	S("2012-12-24 14:30 -01:00"),
> +	S("2012-12-24 16:30:00 +01:00"),
> +	S("2012-12-24 14:30:00 -01:00"),
> +	S("2012-12-24 16:30:00.123456 +01:00"),
> +	S("2012-12-24 14:30:00.123456 -01:00"),
> +	S("2012-12-24 15:30:00.123456 -00:00"),
> +	S("20121224T1630+01:00"),
> +	S("2012-12-24T1630+01:00"),
> +	S("20121224T16:30+01"),
> +	S("20121224T16:30 +01"),
> +};
> +#undef S
> +
> +#define DIM(a) (sizeof(a) / sizeof(a[0]))
> +
> +// p5-time-moment/src/moment_parse.c: parse_string_lenient()

28. Please, stop using // comment opening.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build
  2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-08-05  8:55     ` Safin Timur via Tarantool-patches
  2021-08-08 14:34       ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-05  8:55 UTC (permalink / raw)
  To: Vladislav Shpilevoy, olegrok; +Cc: tarantool-patches

Hello Vlad!

On 05.08.2021 2:58, Vladislav Shpilevoy wrote:
> Hi! Thanks for the patch!
> 
> On 02.08.2021 02:40, Timur Safin via Tarantool-patches wrote:
>> * Integrated chansen/c-dt parser as 3rd party module to the
>>    Tarantool cmake build process.
>> * Points to tsafin/c-dt instead iof original chansen/c-dt to
>>    have easier build integration, because there is additional
>>    commit which integrated cmake support
> 
> As I said in the previous review, it points at
> https://github.com/tarantool/c-dt.git, not at tsafin/c-dt. Why
> do you keep saying the contrary? Look:
> 
> 	url = https://github.com/tarantool/c-dt.git
> 
> It is 'tarantool', not 'tsafin' here.

Yes, that was oversight, due to several restarts of rebase session
(I guess due to mess generated with combination of --autosquash and git 
rerere)
Now finally applied to this commit in branch as commit 
#4b90de3bb21b9f28bc7c1f8e851d4c95f1b0f191.

> 
> Also, like on the previous review, when I call 'make' and do
> 'git status', I see:
> 
> 	Changes not staged for commit:
> 	  (use "git add <file>..." to update what will be committed)
> 	  (use "git restore <file>..." to discard changes in working directory)
> 	  (commit or discard the untracked or modified content in submodules)
> 		modified:   third_party/c-dt (modified content, untracked content)
> 
> You still didn't fix it. Please, do.
> 

That was not a problem for me, due to the fact I _always_ use 
out-of-source build, and all artifacts generated separately under 
${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt/, and such artifacts
inside of source tree problem observed only when you do (not very 
idiomatic) in-source build. cmake generated Makefile overwrite original 
c-dt Makefile.

[While we are here, could you please remind me why we recommend to use 
in-source build in our documentation?
And not prefer the recommended and used by majority of industry for what 
matter out-of-source build?]

We could rather may not influence how and where artifacts will be 
generated from inside of our own c-dt/CMakeLists.txt, we should rather 
modify the way how it get used, like, not include it via simple 
add_subdirectory(which is exactly defining this 
${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt, which would be equal to 
${CMAKE_CURRENT_SOURCE_DIR}/third_party/c-dt for in-source build, but 
rather use some ExternalProject_Add() techniques, which allows to 
redefine generated and binary files localtion. Let me check what is the 
easiest way here...

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-08-06  0:23     ` Safin Timur via Tarantool-patches
  2021-08-06  1:30       ` Oleg Babin via Tarantool-patches
  2021-08-08 11:26       ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 2 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06  0:23 UTC (permalink / raw)
  To: Vladislav Shpilevoy, olegrok; +Cc: tarantool-patches


Hello Vlad,

On 05.08.2021 2:58, Vladislav Shpilevoy wrote:
> Thanks for the patch!
> 
> See 28 comments below.
> 
> 
>> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
>> new file mode 100644
>> index 000000000..84d6642ab
>> --- /dev/null
>> +++ b/src/lib/core/datetime.h

>> +/**
>> + * Full datetime structure representing moments
>> + * since Unix Epoch (1970-01-01).
>> + * Time is kept normalized to UTC, time-zone offset
>> + * is informative only.
>> + */
>> +struct datetime_t {
> 
> 1. Well. How can we proceed beyond the second commit, if you keep
> ignoring my comments? Please, drop _t suffix. Please, comply with
> our code style for comments (they should not be on the same line as
> the members, and should not use any signs like '<').
> 
> If you did it in some later commits - it is not atomic anymore. The
> change must be self-sufficient. This commit is not. It depends on
> one of the last commits which renames something. Why couldn't you
> give the proper names from the beginning, from this commit?

Yes, at that moment renames were in different commit as later 
refactoring. Now, I've squashed those changes to the original commit 
introducing header file, and module in general.

> 
> And what is worse - this entire file is not needed in this commit,
> I already said it before. It is simply not used here at all. Please,
> move it to the relevant commit.
> 
> What is even further worse - you added here some functions which
> do not have any definition. You only added .h file. All the functions
> here are simply undefined. Wtf?
> 
>> +	int64_t secs;	/**< seconds since epoch */
>> +	int32_t nsec;	/**< nanoseconds if any */
>> +	int32_t offset; /**< offset in minutes from UTC */
>> +};
>> +
>> +/**
>> + * Date/time interval structure
>> + */
>> +struct datetime_interval_t {
>> +	int64_t secs; /**< relative seconds delta */
>> +	int32_t nsec; /**< nanoseconds delta */
>> +};
>> +
>> +int
>> +datetime_compare(const struct datetime_t * lhs,
>> +		 const struct datetime_t * rhs);
>> +
>> +
>> +struct datetime_t *
>> +datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
>> +
>> +/**
>> + * Pack datetime_t data to the MessagePack buffer.
>> + */
>> +char *
>> +datetime_pack(char *data, const struct datetime_t *date);
>> +
>> +/**
>> + * Calculate size of MessagePack buffer for datetime_t data.
>> + */
>> +uint32_t
>> +mp_sizeof_datetime(const struct datetime_t *date);
>> +
>> +/**
>> + * Decode data from MessagePack buffer to datetime_t structure.
>> + */
>> +struct datetime_t *
>> +mp_decode_datetime(const char **data, struct datetime_t *date);
>> +
>> +/**
>> + * Encode datetime_t structure to the MessagePack buffer.
>> + */
>> +char *
>> +mp_encode_datetime(char *data, const struct datetime_t *date);
>> +
>> +/**
>> + * Convert datetime to string using default format
>> + * @param date source datetime value
>> + * @param buf output character buffer
>> + * @param len size ofoutput buffer
>> + */
>> +int
>> +datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
>> +
>> +int
>> +mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
>> +
>> +int
>> +mp_fprint_datetime(FILE *file, const char **data, uint32_t len);
> 
> 2. Please, try to follow decimal and uuid here. They have separate files
> for working with MessagePack: tt_uuid.h/mp_uuid.h, decimal.h/mp_decimal.h.
> You should have datetime.h and mp_datetime.h.
> 
> Also once you add these to_string, mp_fprint, mp_snprint, pack/unpack,
> you need to add unitests in C for all these functions. See how UUID does
> it in test/unit/uuid.c and decimal in test/unit/decimal.c.


> 
>> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
>> new file mode 100644
>> index 000000000..870edc541
>> --- /dev/null
>> +++ b/src/lua/datetime.lua
> 
> <...>
> 
>> +    // /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h
>> +    /* ISO C `broken-down time' structure.  */
>> +    struct tm
>> +    {
>> +        int tm_sec;	        /* Seconds.	[0-60] (1 leap second) */
>> +        int tm_min;	        /* Minutes.	[0-59] */
>> +        int tm_hour;        /* Hours.	[0-23] */
>> +        int tm_mday;        /* Day.		[1-31] */
>> +        int tm_mon;	        /* Month.	[0-11] */
>> +        int tm_year;        /* Year	- 1900.  */
>> +        int tm_wday;        /* Day of week.	[0-6] */
>> +        int tm_yday;        /* Days in year.[0-365]	*/
>> +        int tm_isdst;       /* DST.		[-1/0/1]*/
>> +
>> +        long int tm_gmtoff; /* Seconds east of UTC.  */
>> +        const char *tm_zone;/* Timezone abbreviation.  */
> 
> 3. Didn't we already discuss it? The structure is platform-specific.
> You can't use it safely via FFI. And you even agreed with that, but
> still the struct is here. Why?
> 

At that moment C implementation of datetime_strftime, datetime_ctime, 
datetime_asctime, and datetime_now was in different patch in patchset. 
Now it's squashed to the 2nd commit in branch - where module introduced.

>> +    };
> 
> <...>
> 
>> +local datetime_t = ffi.typeof('struct datetime_t')
>> +local interval_t = ffi.typeof('struct datetime_interval_t')
>> +
>> +local function is_interval(o)
>> +    return type(o) == 'cdata' and ffi.istype(interval_t, o)
>> +end
>> +
>> +local function is_datetime(o)
>> +    return type(o) == 'cdata' and ffi.istype(datetime_t, o)
>> +end
>> +
>> +local function is_date_interval(o)
>> +    return is_datetime(o) or is_interval(o)
> 
> 4. You make type(o) == 'cdata' check 2 times here.

Yes. It's not a problem, IMHO. LuaJIT after inlining of those 2 
functions should take care of that. (I believe)

> 
>> +end
>> +
>> +local function interval_new()
>> +    local interval = ffi.new(interval_t)
>> +    return interval
>> +end
>> +
>> +local function check_date(o, message, lvl)
> 
> 5. 'lvl' is never used by any of the invocations. You waste time on
> the 'if', so it can be dropped. Or start using 'lvl' but never pass it
> as nil, please. The same for all the other functions below taking
> 'lvl'.

Cleaned up.

> 
>> +    if lvl == nil then
>> +        lvl = 2
>> +    end
>> +    if not is_datetime(o) then
>> +        return error(("%s: expected datetime, but received %s"):
>> +                     format(message, o), lvl)
>> +    end
>> +end
> <...>
> 
>> +
>> +local function datetime_cmp(lhs, rhs)
> 
> 6. This does not look correct:
> 
> tarantool> datetime = require('datetime')
> ---
> ...
> 
> tarantool> d = datetime()
> ---
> ...
> 
> tarantool> d < 'bad string'
> ---
> - false
> ...
> 
> tarantool> d > 'bad string'
> ---
> - false
> ...

These results in 1st commit were actually side-effect of incorrect 
comparison implementation. But even today, after all interval extensions 
added and all extra checks introduced, it still shows same results for 
bogus data of unsupported types.
==========================================
local function datetime_cmp(lhs, rhs)
     if not is_date_interval(lhs) or
        not is_date_interval(rhs) then
        return nil
     end
     local sdiff = lhs.secs - rhs.secs
     return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
end

local function datetime_eq(lhs, rhs)
     local rc = datetime_cmp(lhs, rhs)
     return rc ~= nil and rc == 0 or false
end

local function datetime_lt(lhs, rhs)
     local rc = datetime_cmp(lhs, rhs)
     return rc ~= nil and rc < 0 or false
end

local function datetime_le(lhs, rhs)
     local rc = datetime_cmp(lhs, rhs)
     return rc ~= nil and rc <= 0 or false
end

==========================================
i.e. whenever we see not datetime or interva value datetime_cmp return 
nil, and all comparisons always return false. Please give me know if 
that's incorrect scenario and we should raise exception. This was the 
way Oleg Bain has requested it to behave. (return false for bogus data)

> 
> Do you have any tests? In this commit I see zero tests for
> your code. Only very few tests for dt library. Please, add
> tests for everything you have here.

A lot. It started to be added with the next commit, and added with any 
other commit adding functionality to datetime.lua. It's pity you didn't 
have a chance to see next commits.

> 
>> +    if not is_date_interval(lhs) or
>> +       not is_date_interval(rhs) then
>> +       return nil
>> +    end
>> +    local sdiff = lhs.secs - rhs.secs
>> +    return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
>> +end
>> +
>> +local function datetime_eq(lhs, rhs)
>> +    local rc = datetime_cmp(lhs, rhs)
>> +    return rc ~= nil and rc == 0 or false
>> +end
>> +
>> +local function datetime_lt(lhs, rhs)
>> +    local rc = datetime_cmp(lhs, rhs)
>> +    return rc ~= nil and rc < 0 or false
>> +end
>> +
>> +local function datetime_le(lhs, rhs)
>> +    local rc = datetime_cmp(lhs, rhs)
>> +    return rc ~= nil and rc <= 0 or false
>> +end
>> +
>> +local function datetime_serialize(self)
>> +    -- Allow YAML, MsgPack and JSON to dump objects with sockets
>> +    return { secs = self.secs, nsec = self.nsec, offset = self.offset }
>> +end
>> +
>> +local function interval_serialize(self)
>> +    -- Allow YAML and JSON to dump objects with sockets
> 
> 7. What do these comments mean? Why is it specific to these
> formats? And why doesn this comment include MsgPack while the
> previous one does?

Removed confusing text.

> 
>> +    return { secs = self.secs, nsec = self.nsec }
>> +end
>> +
>> +local function local_rd(o)
>> +    return math.floor(tonumber(o.secs / SECS_PER_DAY)) + DT_EPOCH_1970_OFFSET
> 
> 8. I suggest to cache math.floor into a global variable. The same
> for the other math.* usages in this file. To save one index operation.

Cached math.floor and math.modf.

> 
>> +end
>> +
>> +local function local_dt(o)
>> +    return builtin.dt_from_rdn(local_rd(o))
>> +end
>> +
>> +local function _normalize_nsec(secs, nsec)
> 
> 9. Why do you have '_' prefix? The function is not global, you
> don't need any protection against name conflicts.

Removed _ from the name.

> 
>> +    if nsec < 0 then
>> +        secs = secs - 1
>> +        nsec = nsec + NANOS_PER_SEC
>> +    elseif nsec >= NANOS_PER_SEC then
>> +        secs = secs + 1
>> +        nsec = nsec - NANOS_PER_SEC
>> +    end
>> +    return secs, nsec
>> +end
>> +
>> +local function check_range(v, range, txt)
>> +    assert(#range == 2)
>> +    if not (v >= range[1] and v <= range[2]) then
> 
> 10. Wouldn't it be shorter to make
> 
> 	if v < range[1] or v > range[2] then
 >
 > ?
 >

Applied.

>> +        error(('value %d of %s is out of allowed range [%d, %d]'):
>> +              format(v, txt, range[1], range[2]))
>> +    end
>> +end
>> +
>> +local datetime_index_handlers = {
>> +    unixtime = function(self)
>> +        return self.secs
>> +    end,
>> +
>> +    timestamp = function(self)
>> +        return tonumber(self.secs) + self.nsec / 1e9
> 
> 11. If you are saying the Lua number value range is enough,
> why do you store them as 64bit integers in 'self' and
> convert to a number only for the external API?

May be I misunderstood your sentence, but let me elaborate here.
I need seconds and nanoseconds kept separately for their efficient and 
precise handling, and for passing to c-dt.

If we would keep 32-bit integers in seconds then we would be able to 
handle only dates upto 2038 year, thus we need 64-bit seconds for 
extended range.

OTOH, not whole 64-bit range is practical, and required for expected in 
real-life datetime range values. It's not a problem that Lua number and 
int64_t have very different range for precise integer values. Lua number 
could handle integer values upto 9e15, which is corresponding to ...

	tarantool> T.secs = 9e15
	---
	...
	
	tarantool> T
	---
	- 2979311-04-03T16:00Z
	...

Which is more than enough for us.

> 
>> +    end,
>> +
>> +    nanoseconds = function(self)
>> +        return self.secs * 1e9 + self.nsec
>> +    end,
>> +
>> +    microseconds = function(self)
>> +        return self.secs * 1e6 + self.nsec / 1e3
>> +    end,
>> +
>> +    milliseconds = function(self)
>> +        return self.secs * 1e3 + self.nsec / 1e6
>> +    end,
>> +
>> +    seconds = function(self)
>> +        return tonumber(self.secs) + self.nsec / 1e9
>> +    end,
>> +
>> +    minutes = function(self)
>> +        return (tonumber(self.secs) + self.nsec / 1e9) / 60
>> +    end,
>> +
>> +    hours = function(self)
>> +        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60)
>> +    end,
>> +
>> +    days = function(self)
>> +        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60) / 24
> 
> 12. Is Lua parser able to merge the last constants into one
> number to make only one division at runtime? If not, I suggest
> to do it manually.

Good question. Apparently Lua parser is able to fold constant expression 
if it's inside of parenthesis, and is unable to cross pairs boundaries. 
Hope LuaJIT in -jO2 mode is better here, but at least we have a 
workaround for constant expressions - put them in the same pairs.

i.e.
```
local days = function(self)
        return (tonumber(self.secs) + self.nsec / 1e9) / (24 * 60 * 60)
end
```

calculates constant expression value (see 86400)
```
-- BYTECODE -- days_ffi.lua:9-11
0001    GGET     1   0      ; "tonumber"
0002    TGETS    2   0   1  ; "secs"
0003    CALL     1   2   2
0004    TGETS    2   0   2  ; "nsec"
0005    DIVVN    2   2   0  ; 1000000000
0006    ADDVV    1   1   2
0007    DIVVN    1   1   1  ; 86400
0008    RET1     1   2
```

but original version:
```
local days = function(self)
        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60) / 24
end
```
had separate expression leafs, without folded constant:
```
-- BYTECODE -- days_ffi.lua:9-11
0001    GGET     1   0      ; "tonumber"
0002    TGETS    2   0   1  ; "secs"
0003    CALL     1   2   2
0004    TGETS    2   0   2  ; "nsec"
0005    DIVVN    2   2   0  ; 1000000000
0006    ADDVV    1   1   2
0007    DIVVN    1   1   1  ; 3600
0008    DIVVN    1   1   2  ; 24
0009    RET1     1   2
```

Changed to use parenthesis accordingly.

> 
>> +    end,
>> +}
>> +
>> +local datetime_index = function(self, key)
>> +    return datetime_index_handlers[key] ~= nil and
>> +           datetime_index_handlers[key](self) or nil
> 
> 13. To save one hash lookup operation you can do this:
> 
> 	local h = datetime_index_handlers[key]
> 	return h and h(self)
> 
> The same for datetime_newindex.

Applied.

> 
>> +end
>> +
>> +local datetime_newindex_handlers = {
>> +    unixtime = function(self, value)
>> +        self.secs = value
>> +        self.nsec, self.offset = 0, 0
>> +    end,
>> +
>> +    timestamp = function(self, value)
>> +        local secs, frac = math.modf(value)
>> +        self.secs = secs
>> +        self.nsec = frac * 1e9
>> +        self.offset = 0
>> +    end,
>> +}
>> +
>> +local function datetime_newindex(self, key, value)
>> +    if datetime_newindex_handlers[key] ~= nil then
>> +        datetime_newindex_handlers[key](self, value)
>> +    end
>> +end
>> +
>> +local function datetime_new_raw(secs, nsec, offset)
>> +    local dt_obj = ffi.new(datetime_t)
>> +    dt_obj.secs = secs
>> +    dt_obj.nsec = nsec
>> +    dt_obj.offset = offset
>> +    return dt_obj
>> +end
>> +
>> +local function mk_timestamp(dt, sp, fp, offset)
> 
> 14. What are 'sp' and 'fp'?

sp - seconds part
fp - fractional part

Agreed that those names were confusing, so renamed to simply `secs` and 
`frac`.

> 
> 15. The function is a datetime constructor, because it creates and
> returns a new datetime object. Please, call it correspondingly
> then. For instance, `datetime_new_dt()`.

Renamed to datetime_new_dt

> 
>> +    local epochV = dt ~= nil and (builtin.dt_rdn(dt) - DT_EPOCH_1970_OFFSET) *
>> +                   SECS_PER_DAY or 0
>> +    local spV = sp ~= nil and sp or 0
>> +    local fpV = fp ~= nil and fp or 0
>> +    local ofsV = offset ~= nil and offset or 0
>> +    return datetime_new_raw (epochV + spV - ofsV * 60, fpV, ofsV)
>> +end
>> +
>> +-- create @datetime_t given object @o fields
> 
> 16. Didn't we agree in the previous review that you will stop using
> name 'o'? At least in the context where there are zeros. It is very
> confusing.
> 

Renamed this confusing case to `obj`.

>> +local function datetime_new(o)
>> +    if o == nil then
>> +        return datetime_new_raw(0, 0, 0)
>> +    end
>> +    local secs = 0
>> +    local nsec = 0
>> +    local offset = 0
>> +    local easy_way = false
>> +    local y = 0
>> +    local M = 0
>> +    local d = 0
>> +    local ymd = false
>> +
>> +    local h = 0
>> +    local m = 0
>> +    local s = 0
>> +    local frac = 0
>> +    local hms = false
>> +
>> +    local dt = 0
>> +
>> +    local handlers = {
> 
> 17. Please, stop using closures in the constructor of a
> potentionally very frequently created/deleted object >

I still disagreed that datetime_new constructor will be on critical 
path, the most frequent case expected to be parse*() -> 
datetime_new_dt() -> datetime_new_raw(). i.e. when we parse millions of 
timestamps in logs and then proceed them.

datetime_new(builder_object) expected to be called in rare cases when 
object got created from Lua code, not on performance critical paths, 
thus convenience here is more important than performance.

And BTW I've created simple benchmark for comparing datetime_new_dt vs 
datetime_new(from builder object) vs datetime_new_2 which would be not 
using hash object with handlers, but rather sequence of ifs.

https://gist.github.com/tsafin/31cc9b0872b6015904fcc90d97740770
```
  ~/datetime/tarantoolt [tsafin/gh-5941-datetime-v3]
00:39 $ ./build/src/tarantool ../bench-hash-dt.lua
datetime_new_dt 0.002718448638916
datetime_new    5.1318607330322
datetime_new(precreated)        4.4448664188385
datetime_new_2(ifs)     4.4977836608887
```

Note, that creation from builder object is much, much slower than raw 
cdata object creation (that's expected). If we move object out of loop, 
and precreate it then we save some time (as expected) 5.1 vs 4.4.
But if we change implementation from hash with closure handlers to 
sequence of ifs - we actually degrade performance (a bit) from 4.44 to 4.49.

Biggest performance impact here - traversing of object attributes, but 
whether it's using closures or not using them, it makes no much difference.

But using closures makes this code look much more elegant. So keeping it 
here for today.


>> +        secs = function(_, v)
>> +            secs = v
>> +            easy_way = true
>> +        end,
>> +
>> +        nsec = function(_, v)
>> +            nsec = v
>> +            easy_way = true
>> +        end,
>> +
>> +        offset = function (_, v)
>> +            offset = v
>> +            easy_way = true
>> +        end,
>> +
>> +        year = function(k, v)
>> +            check_range(v, {1, 9999}, k)
>> +            y = v
>> +            ymd = true
>> +        end,
>> +
>> +        month = function(k, v)
>> +            check_range(v, {1, 12}, k)
>> +            M = v
>> +            ymd = true
>> +        end,
>> +
>> +        day = function(k, v)
>> +            check_range(v, {1, 31}, k)
>> +            d = v
>> +            ymd = true
>> +        end,
>> +
>> +        hour = function(k, v)
>> +            check_range(v, {0, 23}, k)
>> +            h = v
>> +            hms = true
>> +        end,
>> +
>> +        minute = function(k, v)
>> +            check_range(v, {0, 59}, k)
>> +            m = v
>> +            hms = true
>> +        end,
>> +
>> +        second = function(k, v)
>> +            check_range(v, {0, 60}, k)
>> +            s, frac = math.modf(v)
>> +            frac = frac * 1e9 -- convert fraction to nanoseconds
>> +            hms = true
>> +        end,
>> +
>> +        -- tz offset in minutes
>> +        tz = function(k, v)
>> +            check_range(v, {0, 720}, k)
>> +            offset = v
>> +        end
>> +    }
>> +    for key, value in pairs(o) do
>> +        handlers[key](key, value)
>> +    end
>> +
>> +    -- .sec, .nsec, .offset
>> +    if easy_way then
>> +        return datetime_new_raw(secs, nsec, offset)
>> +    end
>> +
>> +    -- .year, .month, .day
>> +    if ymd then
>> +        dt = dt + builtin.dt_from_ymd(y, M, d)
>> +    end
>> +
>> +    -- .hour, .minute, .second
>> +    if hms then
>> +        secs = h * 3600 + m * 60 + s
>> +    end
>> +
>> +    return mk_timestamp(dt, secs, frac, offset)
>> +end
> 
> 18. It fails with a bad error:
> 
> tarantool> d = datetime({garbage = true})
> ---
> - error: 'builtin/datetime.lua:449: attempt to call a nil value'
> ...
> 

Added default handler for an unknown attribute.

```
tarantool> T = date{years = -123}
---
- error: '[string "T = date{years = -123}"]:1: unknown attribute years'
...
```

> 
> 19. It can ignore some fields (here it ignores 'year'):
> 
> tarantool> d1 = datetime({secs = 100, year = 1})
> ---
> ...
> 
> tarantool> d2 = datetime({secs = 100, year = 2})
> ---
> ...
> 
> tarantool> d1 == d2
> ---
> - true
> ...
> 

Yes, and that's by design. .secs defines 'easy_way' of initializing 
object via secs/nsec/offset, while .year selects 'ymd' mode. And 
`easy_way` has precedence. It looks reasonable for me, please, suggest 
your idea how to handle it even better without much redesign?

> 
> 20. Serialization does not seem to be working in the console
> despite you tried to implement it above:
> 
> tarantool> d = datetime({secs = 100})
> ---
> ...
> 
> tarantool> d
> ---
> - 'cdata<struct datetime_t>: 0x0103ad9c50'
> ...

Yup, redefining of __serialize is not enough for cdata object by 
whatever reason (I didn't look deep). But after the next patch(es) it's 
start to stringize properly.

> 
>> +
>> +local function date_first(lhs, rhs)
>> +    if is_datetime(lhs) then
>> +        return lhs, rhs
>> +    else
>> +        return rhs, lhs
>> +    end
>> +end
>> +
>> +local function error_incompatible(name)
>> +    error(("datetime:%s() - incompatible type of arguments"):
>> +          format(name), 3)
>> +end
>> +
>> +local function datetime_sub(lhs, rhs)
>> +    check_date_interval(lhs, "operator -")
>> +    local d, s = lhs, rhs
>> +    local left_t = ffi.typeof(d)
>> +    local right_t = ffi.typeof(s)
>> +    local o
>> +
>> +    if left_t == datetime_t then
>> +        -- left is date, right is date or generic interval
>> +        if (right_t == datetime_t or right_t == interval_t) then
> 
> 21. You do not need () around the condition in Lua.

Fixed.

> 
>> +            o = right_t == datetime_t and interval_new() or datetime_new()
>> +            o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
>> +                                            lhs.nsec - rhs.nsec)
>> +            return o
>> +        else
>> +            error_incompatible("operator -")
>> +        end
>> +    -- both left and right are generic intervals
>> +    elseif left_t == interval_t and right_t == interval_t then
>> +        o = interval_new()
>> +        o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
>> +                                        lhs.nsec - rhs.nsec)
>> +        return o
>> +    else
>> +        error_incompatible("operator -")
>> +    end
>> +end
> 
> 22. Did you really test this function?
> 
> tarantool> dt = require('datetime')
> ---
> ...
> 
> tarantool> d1 = dt()
> ---
> ...
> 
> tarantool> d2 - 1
> ---
> - error: 'builtin/datetime.lua:487: bad argument #1 to ''typeof'' (C type expected,
>      got number)'
> ...

All kinds of arguments checking implemented in the separate patch about 
interval arithmetic operations, please find a moment to look into them.

> 
>> +
>> +local function datetime_add(lhs, rhs)
>> +    local d, s = date_first(lhs, rhs)
>> +
>> +    check_date_interval(d, "operator +")
>> +    check_interval(s, "operator +")
>> +    local left_t = ffi.typeof(d)
>> +    local right_t = ffi.typeof(s)
>> +    local o
>> +
>> +    -- left is date, right is date or interval
>> +    if left_t == datetime_t and right_t == interval_t then
>> +        o = datetime_new()
>> +        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
>> +        return o
>> +    -- both left and right are generic intervals
>> +    elseif left_t == interval_t and right_t == interval_t then
>> +        o = interval_new()
>> +        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
>> +        return o
>> +    else
>> +        error_incompatible("operator +")
>> +    end
>> +end
>> +
>> +-- simple parse functions:
>> +-- parse_date/parse_time/parse_zone
> 
> 23. As I said in the previous review, the comment is useless.
> You just listed 3 function names following the comment. What
> is the purpose of that? What should I learn from it? Please,
> drop it.

Dropped.

> 
> <...>
> 
>> +
>> +local function datetime_from(o)
>> +    if o == nil or type(o) == 'table' then
>> +        return datetime_new(o)
>> +    elseif type(o) == 'string' then
>> +        return parse(o)
>> +    end
>> +end
>> +
>> +local function local_now()
>> +    local p_tv = ffi.new('struct timeval [1]')
>> +    local rc = builtin.gettimeofday(p_tv, nil)
> 
> 24. Why can't you use fiber time? It should be enough in
> nearly all usecases and is much faster. For real-real time
> talking directly to the kernel we have 'clock' Lua module
> already. You could check if results of clock functions can
> be passed into your module to create 'real' timestamps not
> depending on the eventloop.

Fiber time, which actually return cached ev_rt_now, looks imprecise and 
depends on event loop circumstances, though eventually it calls the same 
clock_gettime(). So fiber-time is looking like a cached timestamp value 
from [hopefully, non-distant] past.

I'd rather prefer to have more realtime values, and 
clock_realtime*/clock_monotonic* look relevant, they call 
clock_gettime() for their purposes, which in turn calls the same 
gettimeofday(). So in this case we call same api, but with 2 extra call 
layers above.

IMHO, not worth extra complexity if we could just simply call 
gettimeofday().

[BTW, in current branch this code lives in C - I've moved 
ctime/asctime/strftime/now handlers to datetime.c, to hide Linux vs BSD 
differences there]

> 
>> +    assert(rc == 0)
>> +
>> +    local secs = p_tv[0].tv_sec
>> +    local nsec = p_tv[0].tv_usec * 1000
>> +
>> +    local p_time = ffi.new('time_t[1]')
>> +    local p_tm = ffi.new('struct tm[1]')
>> +    builtin.time(p_time)
>> +    builtin.localtime_r(p_time, p_tm)
>> +    local ofs = p_tm[0].tm_gmtoff / 60 -- convert seconds to minutes
>> +
>> +    return datetime_new_raw(secs, nsec, ofs)
>> +end
>> +
>> +local function datetime_to_tm_ptr(o)
>> +    assert(is_datetime(o))
>> +    local p_tm = ffi.new('struct tm[1]')
>> +    -- dt_to_struct_tm() fills only date data
>> +    builtin.dt_to_struct_tm(local_dt(o), p_tm)
>> +
>> +    -- calculate the smaller data (hour, minute,
>> +    -- seconds) using datetime seconds value
>> +    local seconds_of_day = o.secs % 86400
>> +    local hour = (seconds_of_day / 3600) % 24
>> +    local minute = (seconds_of_day / 60) % 60
>> +    p_tm[0].tm_sec = seconds_of_day % 60
>> +    p_tm[0].tm_min = minute
>> +    p_tm[0].tm_hour = hour
>> +
>> +    p_tm[0].tm_gmtoff = o.offset * 60
>> +
>> +    return p_tm
>> +end

There is no such code in Lua anymore.

>> +
>> +local function asctime(o)
>> +    check_date(o, "datetime:asctime()")
>> +
>> +    local p_tm = datetime_to_tm_ptr(o)
>> +    return ffi.string(builtin.asctime(p_tm))
>> +end
>> +
>> +local function ctime(o)
>> +    check_date(o, "datetime:ctime()")
>> +    local p_time = ffi.new('time_t[1]')
>> +    p_time[0] = o.secs
>> +    return ffi.string(builtin.ctime(p_time))
> 
> 25. Functions like ctime can't be used in Lua, because according
> to their Linux man:
> 
> 	The return value points to a statically allocated string
> 	which might be overwritten by subsequent calls to any of the
> 	date and time functions
> 
> Which means they use a global buffer. If you will
> use datetime module in any of __gc handlers, it will
> return garbage sometimes or just override results
> somethere. That is exactly the same problem as existed
> with tarantool's static buffer.

I do not believe it's the same problem - in this case content of static 
buffer is immediately interned in Lua via ffi.string()

https://luajit.org/ext_ffi_api.html
"The Lua string is an (interned) copy of the data and bears no relation 
to the original data area anymore."

> 
>> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
>> new file mode 100644
>> index 000000000..b6f568c03
>> --- /dev/null
>> +++ b/test/unit/datetime.c
> 
> 26. This file tests dt library. It does not test your code. Please, add
> tests for everything you implemented in this commit. For c-dt library
> the tests must be added in the previous commit, and the test file
> must be called test/unit/cdt.c then.

Unit-test has been moved to the commit introducing c-dt. And in turn, 
initial datetime test (checking parser and ctime/asctime/now) squashed 
to the same commit where we introduce module with parser.

> 
>> @@ -0,0 +1,221 @@
>> +#include "dt.h"
>> +#include <assert.h>
>> +#include <stdint.h>
>> +#include <string.h>
>> +#include <time.h>
>> +
>> +#include "unit.h"
>> +
>> +const char sample[] = "2012-12-24T15:30Z";
> 
> 27. Please, make it static.

Done

> 
>> +
>> +#define S(s) {s, sizeof(s) - 1}
>> +struct {
>> +	const char * sz;
>> +	size_t len;
>> +} tests[] = {
>> +	S("2012-12-24 15:30Z"),
>> +	S("2012-12-24 15:30z"),
>> +	S("2012-12-24 15:30"),
>> +	S("2012-12-24 16:30+01:00"),
>> +	S("2012-12-24 16:30+0100"),
>> +	S("2012-12-24 16:30+01"),
>> +	S("2012-12-24 14:30-01:00"),
>> +	S("2012-12-24 14:30-0100"),
>> +	S("2012-12-24 14:30-01"),
>> +	S("2012-12-24 15:30:00Z"),
>> +	S("2012-12-24 15:30:00z"),
>> +	S("2012-12-24 15:30:00"),
>> +	S("2012-12-24 16:30:00+01:00"),
>> +	S("2012-12-24 16:30:00+0100"),
>> +	S("2012-12-24 14:30:00-01:00"),
>> +	S("2012-12-24 14:30:00-0100"),
>> +	S("2012-12-24 15:30:00.123456Z"),
>> +	S("2012-12-24 15:30:00.123456z"),
>> +	S("2012-12-24 15:30:00.123456"),
>> +	S("2012-12-24 16:30:00.123456+01:00"),
>> +	S("2012-12-24 16:30:00.123456+01"),
>> +	S("2012-12-24 14:30:00.123456-01:00"),
>> +	S("2012-12-24 14:30:00.123456-01"),
>> +	S("2012-12-24t15:30Z"),
>> +	S("2012-12-24t15:30z"),
>> +	S("2012-12-24t15:30"),
>> +	S("2012-12-24t16:30+01:00"),
>> +	S("2012-12-24t16:30+0100"),
>> +	S("2012-12-24t14:30-01:00"),
>> +	S("2012-12-24t14:30-0100"),
>> +	S("2012-12-24t15:30:00Z"),
>> +	S("2012-12-24t15:30:00z"),
>> +	S("2012-12-24t15:30:00"),
>> +	S("2012-12-24t16:30:00+01:00"),
>> +	S("2012-12-24t16:30:00+0100"),
>> +	S("2012-12-24t14:30:00-01:00"),
>> +	S("2012-12-24t14:30:00-0100"),
>> +	S("2012-12-24t15:30:00.123456Z"),
>> +	S("2012-12-24t15:30:00.123456z"),
>> +	S("2012-12-24t16:30:00.123456+01:00"),
>> +	S("2012-12-24t14:30:00.123456-01:00"),
>> +	S("2012-12-24 16:30 +01:00"),
>> +	S("2012-12-24 14:30 -01:00"),
>> +	S("2012-12-24 15:30 UTC"),
>> +	S("2012-12-24 16:30 UTC+1"),
>> +	S("2012-12-24 16:30 UTC+01"),
>> +	S("2012-12-24 16:30 UTC+0100"),
>> +	S("2012-12-24 16:30 UTC+01:00"),
>> +	S("2012-12-24 14:30 UTC-1"),
>> +	S("2012-12-24 14:30 UTC-01"),
>> +	S("2012-12-24 14:30 UTC-01:00"),
>> +	S("2012-12-24 14:30 UTC-0100"),
>> +	S("2012-12-24 15:30 GMT"),
>> +	S("2012-12-24 16:30 GMT+1"),
>> +	S("2012-12-24 16:30 GMT+01"),
>> +	S("2012-12-24 16:30 GMT+0100"),
>> +	S("2012-12-24 16:30 GMT+01:00"),
>> +	S("2012-12-24 14:30 GMT-1"),
>> +	S("2012-12-24 14:30 GMT-01"),
>> +	S("2012-12-24 14:30 GMT-01:00"),
>> +	S("2012-12-24 14:30 GMT-0100"),
>> +	S("2012-12-24 14:30 -01:00"),
>> +	S("2012-12-24 16:30:00 +01:00"),
>> +	S("2012-12-24 14:30:00 -01:00"),
>> +	S("2012-12-24 16:30:00.123456 +01:00"),
>> +	S("2012-12-24 14:30:00.123456 -01:00"),
>> +	S("2012-12-24 15:30:00.123456 -00:00"),
>> +	S("20121224T1630+01:00"),
>> +	S("2012-12-24T1630+01:00"),
>> +	S("20121224T16:30+01"),
>> +	S("20121224T16:30 +01"),
>> +};
>> +#undef S
>> +
>> +#define DIM(a) (sizeof(a) / sizeof(a[0]))
>> +
>> +// p5-time-moment/src/moment_parse.c: parse_string_lenient()
> 
> 28. Please, stop using // comment opening.
> 

Uh, ok.

All, with exception of separation of mp_datetime and introduction of 
test (test requires some time for plumbing) is alreday in the branch 
updated - tsafin/gh-5941-datetime-v3

Vlad, if you still have some time this Friday, before goin to vacation, 
then please look into the rest of patches I have so far. Now they 7, and 
look reasonable enough:

* a1905f727 build: add Christian Hansen c-dt to the build
* f1dcde7ec lua: built-in module datetime
* 88e25eaa0 lua, datetime: display datetime
* cfa40d93e box, datetime: messagepack support for datetime
* 9e718e5c5 box, datetime: datetime comparison for indices
* 8d26d2f76 lua, datetime: time intervals support
* 9b338fc57 datetime: changelog for datetime module

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 3/9] lua, datetime: datetime tests
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 3/9] lua, datetime: datetime tests Timur Safin via Tarantool-patches
@ 2021-08-06  0:25   ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06  0:25 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

This guy got squashed to patch 2/9 - lua: built-in module datetime.

Timur

On 02.08.2021 3:40, Timur Safin wrote:
> * created app-tap test for new builtin module `datetime.lua`
> * added case to check datetime string formatting using:
>    - asctime (gmt time);
>    - ctime (local TZ time);
>    - strftime (using given format).
> 
> * added positive/negative checks to datetime test:
>    - for that,in the datetime.lua, we have extended api of `parse_date()`,
>      `parse_time()`, and `parse_time_zone()` so they return
>      not only parsed object, but also a length of parsed substring;
>    - which allows us to parse even _partially_ valid strings like
>      "20121224 Foo bar".
> 
> * checks calculated attributes to date object, e.g.:
>    - timestamp, seconds, microseconds, minute, or hours
> 
> Part of #5941
> ---
>   test/app-tap/datetime.test.lua | 206 +++++++++++++++++++++++++++++++++
>   1 file changed, 206 insertions(+)
>   create mode 100755 test/app-tap/datetime.test.lua
> 
> diff --git a/test/app-tap/datetime.test.lua b/test/app-tap/datetime.test.lua
> new file mode 100755
> index 000000000..f7541567e
> --- /dev/null
> +++ b/test/app-tap/datetime.test.lua
> @@ -0,0 +1,206 @@
> +#!/usr/bin/env tarantool
> +
> +local tap = require('tap')
> +local test = tap.test("errno")
> +local date = require('datetime')
> +local ffi = require('ffi')
> +
> +
> +test:plan(6)
> +
> +test:test("Simple tests for parser", function(test)
> +    test:plan(2)
> +    test:ok(date("1970-01-01T01:00:00Z") ==
> +            date {year=1970, month=1, day=1, hour=1, minute=0, second=0})
> +    test:ok(date("1970-01-01T02:00:00+02:00") ==
> +            date {year=1970, month=1, day=1, hour=2, minute=0, second=0, tz=120})
> +end)
> +
> +test:test("Multiple tests for parser (with nanoseconds)", function(test)
> +    test:plan(165)
> +    -- borrowed from p5-time-moments/t/180_from_string.t
> +    local tests =
> +    {
> +        { '1970-01-01T00:00:00Z',                       0,           0,    0 },
> +        { '1970-01-01T02:00:00+02:00',                  0,           0,  120 },
> +        { '1970-01-01T01:30:00+01:30',                  0,           0,   90 },
> +        { '1970-01-01T01:00:00+01:00',                  0,           0,   60 },
> +        { '1970-01-01T00:01:00+00:01',                  0,           0,    1 },
> +        { '1970-01-01T00:00:00+00:00',                  0,           0,    0 },
> +        { '1969-12-31T23:59:00-00:01',                  0,           0,   -1 },
> +        { '1969-12-31T23:00:00-01:00',                  0,           0,  -60 },
> +        { '1969-12-31T22:30:00-01:30',                  0,           0,  -90 },
> +        { '1969-12-31T22:00:00-02:00',                  0,           0, -120 },
> +        { '1970-01-01T00:00:00.123456789Z',             0,   123456789,    0 },
> +        { '1970-01-01T00:00:00.12345678Z',              0,   123456780,    0 },
> +        { '1970-01-01T00:00:00.1234567Z',               0,   123456700,    0 },
> +        { '1970-01-01T00:00:00.123456Z',                0,   123456000,    0 },
> +        { '1970-01-01T00:00:00.12345Z',                 0,   123450000,    0 },
> +        { '1970-01-01T00:00:00.1234Z',                  0,   123400000,    0 },
> +        { '1970-01-01T00:00:00.123Z',                   0,   123000000,    0 },
> +        { '1970-01-01T00:00:00.12Z',                    0,   120000000,    0 },
> +        { '1970-01-01T00:00:00.1Z',                     0,   100000000,    0 },
> +        { '1970-01-01T00:00:00.01Z',                    0,    10000000,    0 },
> +        { '1970-01-01T00:00:00.001Z',                   0,     1000000,    0 },
> +        { '1970-01-01T00:00:00.0001Z',                  0,      100000,    0 },
> +        { '1970-01-01T00:00:00.00001Z',                 0,       10000,    0 },
> +        { '1970-01-01T00:00:00.000001Z',                0,        1000,    0 },
> +        { '1970-01-01T00:00:00.0000001Z',               0,         100,    0 },
> +        { '1970-01-01T00:00:00.00000001Z',              0,          10,    0 },
> +        { '1970-01-01T00:00:00.000000001Z',             0,           1,    0 },
> +        { '1970-01-01T00:00:00.000000009Z',             0,           9,    0 },
> +        { '1970-01-01T00:00:00.00000009Z',              0,          90,    0 },
> +        { '1970-01-01T00:00:00.0000009Z',               0,         900,    0 },
> +        { '1970-01-01T00:00:00.000009Z',                0,        9000,    0 },
> +        { '1970-01-01T00:00:00.00009Z',                 0,       90000,    0 },
> +        { '1970-01-01T00:00:00.0009Z',                  0,      900000,    0 },
> +        { '1970-01-01T00:00:00.009Z',                   0,     9000000,    0 },
> +        { '1970-01-01T00:00:00.09Z',                    0,    90000000,    0 },
> +        { '1970-01-01T00:00:00.9Z',                     0,   900000000,    0 },
> +        { '1970-01-01T00:00:00.99Z',                    0,   990000000,    0 },
> +        { '1970-01-01T00:00:00.999Z',                   0,   999000000,    0 },
> +        { '1970-01-01T00:00:00.9999Z',                  0,   999900000,    0 },
> +        { '1970-01-01T00:00:00.99999Z',                 0,   999990000,    0 },
> +        { '1970-01-01T00:00:00.999999Z',                0,   999999000,    0 },
> +        { '1970-01-01T00:00:00.9999999Z',               0,   999999900,    0 },
> +        { '1970-01-01T00:00:00.99999999Z',              0,   999999990,    0 },
> +        { '1970-01-01T00:00:00.999999999Z',             0,   999999999,    0 },
> +        { '1970-01-01T00:00:00.0Z',                     0,           0,    0 },
> +        { '1970-01-01T00:00:00.00Z',                    0,           0,    0 },
> +        { '1970-01-01T00:00:00.000Z',                   0,           0,    0 },
> +        { '1970-01-01T00:00:00.0000Z',                  0,           0,    0 },
> +        { '1970-01-01T00:00:00.00000Z',                 0,           0,    0 },
> +        { '1970-01-01T00:00:00.000000Z',                0,           0,    0 },
> +        { '1970-01-01T00:00:00.0000000Z',               0,           0,    0 },
> +        { '1970-01-01T00:00:00.00000000Z',              0,           0,    0 },
> +        { '1970-01-01T00:00:00.000000000Z',             0,           0,    0 },
> +        { '1973-11-29T21:33:09Z',               123456789,           0,    0 },
> +        { '2013-10-28T17:51:56Z',              1382982716,           0,    0 },
> +        -- { '9999-12-31T23:59:59Z',            253402300799,           0,    0 },
> +    }
> +    for _, value in ipairs(tests) do
> +        local str, epoch, nsec, offset
> +        str, epoch, nsec, offset = unpack(value)
> +        local dt = date(str)
> +        test:ok(dt.secs == epoch, ('%s: dt.secs == %d'):format(str, epoch))
> +        test:ok(dt.nsec == nsec, ('%s: dt.nsec == %d'):format(str, nsec))
> +        test:ok(dt.offset == offset, ('%s: dt.offset == %d'):format(str, offset))
> +    end
> +end)
> +
> +ffi.cdef [[
> +    void tzset(void);
> +]]
> +
> +test:test("Datetime string formatting", function(test)
> +    test:plan(7)
> +    local str = "1970-01-01"
> +    local t = date(str)
> +    test:ok(t.secs == 0, ('%s: t.secs == %d'):format(str, tonumber(t.secs)))
> +    test:ok(t.nsec == 0, ('%s: t.nsec == %d'):format(str, t.nsec))
> +    test:ok(t.offset == 0, ('%s: t.offset == %d'):format(str, t.offset))
> +    test:ok(date.asctime(t) == 'Thu Jan  1 00:00:00 1970\n', ('%s: asctime'):format(str))
> +    -- ctime() is local timezone dependent. To make sure that
> +    -- test is deterministic we enforce timezone via TZ environment
> +    -- manipulations and calling tzset()
> +
> +    -- redefine timezone to be always GMT-2
> +    os.setenv('TZ', 'GMT-2')
> +    ffi.C.tzset()
> +    test:ok(date.ctime(t) == 'Thu Jan  1 02:00:00 1970\n', ('%s: ctime with timezone'):format(str))
> +    test:ok(date.strftime('%d/%m/%Y', t) == '01/01/1970', ('%s: strftime #1'):format(str))
> +    test:ok(date.strftime('%A %d. %B %Y', t) == 'Thursday 01. January 1970', ('%s: strftime #2'):format(str))
> +end)
> +
> +test:test("Parse iso date - valid strings", function(test)
> +    test:plan(32)
> +    local good = {
> +        {2012, 12, 24, "20121224",                   8 },
> +        {2012, 12, 24, "20121224  Foo bar",          8 },
> +        {2012, 12, 24, "2012-12-24",                10 },
> +        {2012, 12, 24, "2012-12-24 23:59:59",       10 },
> +        {2012, 12, 24, "2012-12-24T00:00:00+00:00", 10 },
> +        {2012, 12, 24, "2012359",                    7 },
> +        {2012, 12, 24, "2012359T235959+0130",        7 },
> +        {2012, 12, 24, "2012-359",                   8 },
> +        {2012, 12, 24, "2012W521",                   8 },
> +        {2012, 12, 24, "2012-W52-1",                10 },
> +        {2012, 12, 24, "2012Q485",                   8 },
> +        {2012, 12, 24, "2012-Q4-85",                10 },
> +        {   1,  1,  1, "0001-Q1-01",                10 },
> +        {   1,  1,  1, "0001-W01-1",                10 },
> +        {   1,  1,  1, "0001-01-01",                10 },
> +        {   1,  1,  1, "0001-001",                   8 },
> +    }
> +
> +    for _, value in ipairs(good) do
> +        local year, month, day, str, date_part_len
> +        year, month, day, str, date_part_len = unpack(value)
> +        local expected_date = date{year = year, month = month, day = day}
> +        local date_part, len
> +        date_part, len = date.parse_date(str)
> +        test:ok(len == date_part_len, ('%s: length check %d'):format(str, len))
> +        test:ok(expected_date == date_part, ('%s: expected date'):format(str))
> +    end
> +end)
> +
> +test:test("Parse iso date - invalid strings", function(test)
> +    test:plan(62)
> +    local bad = {
> +        "20121232"   , -- Invalid day of month
> +        "2012-12-310", -- Invalid day of month
> +        "2012-13-24" , -- Invalid month
> +        "2012367"    , -- Invalid day of year
> +        "2012-000"   , -- Invalid day of year
> +        "2012W533"   , -- Invalid week of year
> +        "2012-W52-8" , -- Invalid day of week
> +        "2012Q495"   , -- Invalid day of quarter
> +        "2012-Q5-85" , -- Invalid quarter
> +        "20123670"   , -- Trailing digit
> +        "201212320"  , -- Trailing digit
> +        "2012-12"    , -- Reduced accuracy
> +        "2012-Q4"    , -- Reduced accuracy
> +        "2012-Q42"   , -- Invalid
> +        "2012-Q1-1"  , -- Invalid day of quarter
> +        "2012Q--420" , -- Invalid
> +        "2012-Q-420" , -- Invalid
> +        "2012Q11"    , -- Incomplete
> +        "2012Q1234"  , -- Trailing digit
> +        "2012W12"    , -- Incomplete
> +        "2012W1234"  , -- Trailing digit
> +        "2012W-123"  , -- Invalid
> +        "2012-W12"   , -- Incomplete
> +        "2012-W12-12", -- Trailing digit
> +        "2012U1234"  , -- Invalid
> +        "2012-1234"  , -- Invalid
> +        "2012-X1234" , -- Invalid
> +        "0000-Q1-01" , -- Year less than 0001
> +        "0000-W01-1" , -- Year less than 0001
> +        "0000-01-01" , -- Year less than 0001
> +        "0000-001"   , -- Year less than 0001
> +    }
> +
> +    for _, str in ipairs(bad) do
> +        local date_part, len
> +        date_part, len = date.parse_date(str)
> +        test:ok(len == 0, ('%s: length check %d'):format(str, len))
> +        test:ok(date_part == nil, ('%s: empty date check %s'):format(str, date_part))
> +    end
> +end)
> +
> +test:test("Parse tiny date into seconds and other parts", function(test)
> +    test:plan(9)
> +    local str = '19700101 00:00:30.528'
> +    local tiny = date(str)
> +    test:ok(tiny.secs == 30, ("secs of '%s'"):format(str))
> +    test:ok(tiny.nsec == 528000000, ("nsec of '%s'"):format(str))
> +    test:ok(tiny.nanoseconds == 30528000000, "nanoseconds")
> +    test:ok(tiny.microseconds == 30528000, "microseconds")
> +    test:ok(tiny.milliseconds == 30528, "milliseconds")
> +    test:ok(tiny.seconds == 30.528, "seconds")
> +    test:ok(tiny.timestamp == 30.528, "timestamp")
> +    test:ok(tiny.minutes == 0.5088, "minutes")
> +    test:ok(tiny.hours == 0.00848, "hours")
> +end)
> +
> +os.exit(test:check() and 0 or 1)
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime Timur Safin via Tarantool-patches
  2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-08-06  0:26   ` Safin Timur via Tarantool-patches
  2021-08-08 14:34   ` Vladislav Shpilevoy via Tarantool-patches
  2 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06  0:26 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

These files for unit-test have been moved to the patch 1/9 build: add 
Christian Hansen c-dt to the build


 >   test/unit/CMakeLists.txt  |   2 +
 >   test/unit/datetime.c      | 221 +++++++++++
 >   test/unit/datetime.result | 358 ++++++++++++++++++


On 02.08.2021 3:40, Timur Safin wrote:
> * created a new Tarantool built-in module `datetime`;
> * register cdef types for this module;
> * export some `dt_*` functions from `c-dt` library;
> * lua implementationis of `asctime` and `strftime`;
> * datetime parsing unit tests, with and withput timezones;
> * c test for reversible strftime roundtrip;
> 
> Part of #5941
> ---
>   src/CMakeLists.txt        |   2 +
>   src/exports.h             |  21 ++
>   src/lib/core/datetime.h   | 112 ++++++
>   src/lua/datetime.lua      | 756 ++++++++++++++++++++++++++++++++++++++
>   src/lua/init.c            |   4 +-
>   src/lua/utils.c           |  27 ++
>   src/lua/utils.h           |  12 +
>   test/unit/CMakeLists.txt  |   2 +
>   test/unit/datetime.c      | 221 +++++++++++
>   test/unit/datetime.result | 358 ++++++++++++++++++
>   10 files changed, 1514 insertions(+), 1 deletion(-)
>   create mode 100644 src/lib/core/datetime.h
>   create mode 100644 src/lua/datetime.lua
>   create mode 100644 test/unit/datetime.c
>   create mode 100644 test/unit/datetime.result
> 
> diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
> index ef6a295d5..e0499e57f 100644
> --- a/src/CMakeLists.txt
> +++ b/src/CMakeLists.txt
> @@ -51,6 +51,8 @@ lua_source(lua_sources ../third_party/luafun/fun.lua)
>   lua_source(lua_sources lua/httpc.lua)
>   lua_source(lua_sources lua/iconv.lua)
>   lua_source(lua_sources lua/swim.lua)
> +lua_source(lua_sources lua/datetime.lua)
> +
>   # LuaJIT jit.* library
>   lua_source(lua_sources ${LUAJIT_SOURCE_ROOT}/src/jit/bc.lua)
>   lua_source(lua_sources ${LUAJIT_SOURCE_ROOT}/src/jit/bcsave.lua)
> diff --git a/src/exports.h b/src/exports.h
> index 5bb3e6a2b..2ec314369 100644
> --- a/src/exports.h
> +++ b/src/exports.h
> @@ -218,6 +218,27 @@ EXPORT(curl_version)
>   EXPORT(curl_version_info)
>   #endif /* EXPORT_LIBCURL_SYMBOLS */
>   EXPORT(decimal_unpack)
> +EXPORT(dt_dow)
> +EXPORT(dt_from_rdn)
> +EXPORT(dt_from_struct_tm)
> +EXPORT(dt_from_yd)
> +EXPORT(dt_from_ymd)
> +EXPORT(dt_from_yqd)
> +EXPORT(dt_from_ywd)
> +EXPORT(dt_parse_iso_date)
> +EXPORT(dt_parse_iso_time)
> +EXPORT(dt_parse_iso_time_basic)
> +EXPORT(dt_parse_iso_time_extended)
> +EXPORT(dt_parse_iso_zone)
> +EXPORT(dt_parse_iso_zone_basic)
> +EXPORT(dt_parse_iso_zone_extended)
> +EXPORT(dt_parse_iso_zone_lenient)
> +EXPORT(dt_rdn)
> +EXPORT(dt_to_struct_tm)
> +EXPORT(dt_to_yd)
> +EXPORT(dt_to_ymd)
> +EXPORT(dt_to_yqd)
> +EXPORT(dt_to_ywd)
>   EXPORT(error_ref)
>   EXPORT(error_set_prev)
>   EXPORT(error_unpack_unsafe)
> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> new file mode 100644
> index 000000000..84d6642ab
> --- /dev/null
> +++ b/src/lib/core/datetime.h
> @@ -0,0 +1,112 @@
> +#pragma once
> +/*
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * 1. Redistributions of source code must retain the above
> + *    copyright notice, this list of conditions and the
> + *    following disclaimer.
> + *
> + * 2. Redistributions in binary form must reproduce the above
> + *    copyright notice, this list of conditions and the following
> + *    disclaimer in the documentation and/or other materials
> + *    provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
> + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
> + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
> + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
> + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include "c-dt/dt.h"
> +
> +#if defined(__cplusplus)
> +extern "C"
> +{
> +#endif /* defined(__cplusplus) */
> +
> +/**
> + * Full datetime structure representing moments
> + * since Unix Epoch (1970-01-01).
> + * Time is kept normalized to UTC, time-zone offset
> + * is informative only.
> + */
> +struct datetime_t {
> +	int64_t secs;	/**< seconds since epoch */
> +	int32_t nsec;	/**< nanoseconds if any */
> +	int32_t offset; /**< offset in minutes from UTC */
> +};
> +
> +/**
> + * Date/time interval structure
> + */
> +struct datetime_interval_t {
> +	int64_t secs; /**< relative seconds delta */
> +	int32_t nsec; /**< nanoseconds delta */
> +};
> +
> +int
> +datetime_compare(const struct datetime_t * lhs,
> +		 const struct datetime_t * rhs);
> +
> +
> +struct datetime_t *
> +datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
> +
> +/**
> + * Pack datetime_t data to the MessagePack buffer.
> + */
> +char *
> +datetime_pack(char *data, const struct datetime_t *date);
> +
> +/**
> + * Calculate size of MessagePack buffer for datetime_t data.
> + */
> +uint32_t
> +mp_sizeof_datetime(const struct datetime_t *date);
> +
> +/**
> + * Decode data from MessagePack buffer to datetime_t structure.
> + */
> +struct datetime_t *
> +mp_decode_datetime(const char **data, struct datetime_t *date);
> +
> +/**
> + * Encode datetime_t structure to the MessagePack buffer.
> + */
> +char *
> +mp_encode_datetime(char *data, const struct datetime_t *date);
> +
> +/**
> + * Convert datetime to string using default format
> + * @param date source datetime value
> + * @param buf output character buffer
> + * @param len size ofoutput buffer
> + */
> +int
> +datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
> +
> +int
> +mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
> +
> +int
> +mp_fprint_datetime(FILE *file, const char **data, uint32_t len);
> +
> +#if defined(__cplusplus)
> +} /* extern "C" */
> +#endif /* defined(__cplusplus) */
> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> new file mode 100644
> index 000000000..870edc541
> --- /dev/null
> +++ b/src/lua/datetime.lua
> @@ -0,0 +1,756 @@
> +local ffi = require('ffi')
> +
> +ffi.cdef [[
> +
> +    /*
> +    `c-dt` library functions handles properly both positive and negative `dt`
> +    values, where `dt` is a number of dates since Rata Die date (0001-01-01).
> +
> +    For better compactness of our typical data in MessagePack stream we shift
> +    root of our time to the Unix Epoch date (1970-01-01), thus our 0 is
> +    actually dt = 719163.
> +
> +    So here is a simple formula how convert our epoch-based seconds to dt values
> +        dt = (secs / 86400) + 719163
> +    Where 719163 is an offset of Unix Epoch (1970-01-01) since Rata Die
> +    (0001-01-01) in dates.
> +
> +    */
> +    typedef int dt_t;
> +
> +    // dt_core.h
> +    typedef enum {
> +        DT_MON       = 1,
> +        DT_MONDAY    = 1,
> +        DT_TUE       = 2,
> +        DT_TUESDAY   = 2,
> +        DT_WED       = 3,
> +        DT_WEDNESDAY = 3,
> +        DT_THU       = 4,
> +        DT_THURSDAY  = 4,
> +        DT_FRI       = 5,
> +        DT_FRIDAY    = 5,
> +        DT_SAT       = 6,
> +        DT_SATURDAY  = 6,
> +        DT_SUN       = 7,
> +        DT_SUNDAY    = 7,
> +    } dt_dow_t;
> +
> +    dt_t     dt_from_rdn     (int n);
> +    dt_t     dt_from_yd      (int y, int d);
> +    dt_t     dt_from_ymd     (int y, int m, int d);
> +    dt_t     dt_from_yqd     (int y, int q, int d);
> +    dt_t     dt_from_ywd     (int y, int w, int d);
> +
> +    void     dt_to_yd        (dt_t dt, int *y, int *d);
> +    void     dt_to_ymd       (dt_t dt, int *y, int *m, int *d);
> +    void     dt_to_yqd       (dt_t dt, int *y, int *q, int *d);
> +    void     dt_to_ywd       (dt_t dt, int *y, int *w, int *d);
> +
> +    int      dt_rdn          (dt_t dt);
> +    dt_dow_t dt_dow          (dt_t dt);
> +
> +    // dt_parse_iso.h
> +    size_t dt_parse_iso_date          (const char *str, size_t len, dt_t *dt);
> +
> +    size_t dt_parse_iso_time          (const char *str, size_t len, int *sod, int *nsec);
> +    size_t dt_parse_iso_time_basic    (const char *str, size_t len, int *sod, int *nsec);
> +    size_t dt_parse_iso_time_extended (const char *str, size_t len, int *sod, int *nsec);
> +
> +    size_t dt_parse_iso_zone          (const char *str, size_t len, int *offset);
> +    size_t dt_parse_iso_zone_basic    (const char *str, size_t len, int *offset);
> +    size_t dt_parse_iso_zone_extended (const char *str, size_t len, int *offset);
> +    size_t dt_parse_iso_zone_lenient  (const char *str, size_t len, int *offset);
> +
> +    // dt_tm.h
> +    dt_t    dt_from_struct_tm  (const struct tm *tm);
> +    void    dt_to_struct_tm    (dt_t dt, struct tm *tm);
> +
> +    // <asm-generic/posix_types.h>
> +    typedef long            __kernel_long_t;
> +    typedef unsigned long   __kernel_ulong_t;
> +    // /usr/include/x86_64-linux-gnu/bits/types/time_t.h
> +    typedef long            time_t;
> +
> +
> +    // <time.h>
> +    typedef __kernel_long_t	__kernel_time_t;
> +    typedef __kernel_long_t	__kernel_suseconds_t;
> +
> +    struct timespec {
> +        __kernel_time_t	        tv_sec;     /* seconds */
> +        long                    tv_nsec;    /* nanoseconds */
> +    };
> +
> +    struct timeval {
> +        __kernel_time_t	        tv_sec;	    /* seconds */
> +        __kernel_suseconds_t    tv_usec;    /* microseconds */
> +    };
> +
> +    struct timezone {
> +        int	tz_minuteswest;     /* minutes west of Greenwich */
> +        int	tz_dsttime;	        /* type of dst correction */
> +    };
> +
> +    // /usr/include/x86_64-linux-gnu/sys/time.h
> +    typedef struct timezone * __timezone_ptr_t;
> +
> +    /* Get the current time of day and timezone information,
> +       putting it into *TV and *TZ.  If TZ is NULL, *TZ is not filled.
> +       Returns 0 on success, -1 on errors.
> +
> +       NOTE: This form of timezone information is obsolete.
> +       Use the functions and variables declared in <time.h> instead.  */
> +    int gettimeofday (struct timeval *__tv, struct timezone * __tz);
> +
> +    // /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h
> +    /* ISO C `broken-down time' structure.  */
> +    struct tm
> +    {
> +        int tm_sec;	        /* Seconds.	[0-60] (1 leap second) */
> +        int tm_min;	        /* Minutes.	[0-59] */
> +        int tm_hour;        /* Hours.	[0-23] */
> +        int tm_mday;        /* Day.		[1-31] */
> +        int tm_mon;	        /* Month.	[0-11] */
> +        int tm_year;        /* Year	- 1900.  */
> +        int tm_wday;        /* Day of week.	[0-6] */
> +        int tm_yday;        /* Days in year.[0-365]	*/
> +        int tm_isdst;       /* DST.		[-1/0/1]*/
> +
> +        long int tm_gmtoff; /* Seconds east of UTC.  */
> +        const char *tm_zone;/* Timezone abbreviation.  */
> +    };
> +
> +    // <time.h>
> +    /* Return the current time and put it in *TIMER if TIMER is not NULL.  */
> +    time_t time (time_t *__timer);
> +
> +    /* Format TP into S according to FORMAT.
> +    Write no more than MAXSIZE characters and return the number
> +    of characters written, or 0 if it would exceed MAXSIZE.  */
> +    size_t strftime (char * __s, size_t __maxsize, const char * __format,
> +                     const struct tm * __tp);
> +
> +    /* Parse S according to FORMAT and store binary time information in TP.
> +    The return value is a pointer to the first unparsed character in S.  */
> +    char *strptime (const char * __s, const char * __fmt, struct tm *__tp);
> +
> +    /* Return the `struct tm' representation of *TIMER in UTC,
> +    using *TP to store the result.  */
> +    struct tm *gmtime_r (const time_t * __timer, struct tm * __tp);
> +
> +    /* Return the `struct tm' representation of *TIMER in local time,
> +    using *TP to store the result.  */
> +    struct tm *localtime_r (const time_t * __timer, struct tm * __tp);
> +
> +    /* Return a string of the form "Day Mon dd hh:mm:ss yyyy\n"
> +    that is the representation of TP in this format.  */
> +    char *asctime (const struct tm *__tp);
> +
> +    /* Equivalent to `asctime (localtime (timer))'.  */
> +    char *ctime (const time_t *__timer);
> +
> +]]
> +
> +local builtin = ffi.C
> +
> +local SECS_PER_DAY     = 86400
> +local NANOS_PER_SEC    = 1000000000LL
> +
> +-- c-dt/dt_config.h
> +
> +-- Unix, January 1, 1970, Thursday
> +local DT_EPOCH_1970_OFFSET = 719163LL
> +
> +
> +local datetime_t = ffi.typeof('struct datetime_t')
> +local interval_t = ffi.typeof('struct datetime_interval_t')
> +
> +local function is_interval(o)
> +    return type(o) == 'cdata' and ffi.istype(interval_t, o)
> +end
> +
> +local function is_datetime(o)
> +    return type(o) == 'cdata' and ffi.istype(datetime_t, o)
> +end
> +
> +local function is_date_interval(o)
> +    return is_datetime(o) or is_interval(o)
> +end
> +
> +local function interval_new()
> +    local interval = ffi.new(interval_t)
> +    return interval
> +end
> +
> +local function check_date(o, message, lvl)
> +    if lvl == nil then
> +        lvl = 2
> +    end
> +    if not is_datetime(o) then
> +        return error(("%s: expected datetime, but received %s"):
> +                     format(message, o), lvl)
> +    end
> +end
> +
> +local function check_date_interval(o, message, lvl)
> +    if lvl == nil then
> +        lvl = 2
> +    end
> +    if not (is_datetime(o) or is_interval(o)) then
> +        return error(("%s: expected datetime or interval, but received %s"):
> +                     format(message, o), lvl)
> +    end
> +end
> +
> +local function check_interval(o, message, lvl)
> +    if lvl == nil then
> +        lvl = 2
> +    end
> +    if not is_interval(o) then
> +        return error(("%s: expected interval, but received %s"):
> +                     format(message, o), lvl)
> +    end
> +end
> +
> +local function check_str(s, message, lvl)
> +    if lvl == nil then
> +        lvl = 2
> +    end
> +    if not type(s) == 'string' then
> +        return error(("%s: expected string, but received %s"):
> +                     format(message, s), lvl)
> +    end
> +end
> +
> +local function datetime_cmp(lhs, rhs)
> +    if not is_date_interval(lhs) or
> +       not is_date_interval(rhs) then
> +       return nil
> +    end
> +    local sdiff = lhs.secs - rhs.secs
> +    return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
> +end
> +
> +local function datetime_eq(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc == 0 or false
> +end
> +
> +local function datetime_lt(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc < 0 or false
> +end
> +
> +local function datetime_le(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc <= 0 or false
> +end
> +
> +local function datetime_serialize(self)
> +    -- Allow YAML, MsgPack and JSON to dump objects with sockets
> +    return { secs = self.secs, nsec = self.nsec, offset = self.offset }
> +end
> +
> +local function interval_serialize(self)
> +    -- Allow YAML and JSON to dump objects with sockets
> +    return { secs = self.secs, nsec = self.nsec }
> +end
> +
> +local function local_rd(o)
> +    return math.floor(tonumber(o.secs / SECS_PER_DAY)) + DT_EPOCH_1970_OFFSET
> +end
> +
> +local function local_dt(o)
> +    return builtin.dt_from_rdn(local_rd(o))
> +end
> +
> +local function _normalize_nsec(secs, nsec)
> +    if nsec < 0 then
> +        secs = secs - 1
> +        nsec = nsec + NANOS_PER_SEC
> +    elseif nsec >= NANOS_PER_SEC then
> +        secs = secs + 1
> +        nsec = nsec - NANOS_PER_SEC
> +    end
> +    return secs, nsec
> +end
> +
> +local function check_range(v, range, txt)
> +    assert(#range == 2)
> +    if not (v >= range[1] and v <= range[2]) then
> +        error(('value %d of %s is out of allowed range [%d, %d]'):
> +              format(v, txt, range[1], range[2]))
> +    end
> +end
> +
> +local datetime_index_handlers = {
> +    unixtime = function(self)
> +        return self.secs
> +    end,
> +
> +    timestamp = function(self)
> +        return tonumber(self.secs) + self.nsec / 1e9
> +    end,
> +
> +    nanoseconds = function(self)
> +        return self.secs * 1e9 + self.nsec
> +    end,
> +
> +    microseconds = function(self)
> +        return self.secs * 1e6 + self.nsec / 1e3
> +    end,
> +
> +    milliseconds = function(self)
> +        return self.secs * 1e3 + self.nsec / 1e6
> +    end,
> +
> +    seconds = function(self)
> +        return tonumber(self.secs) + self.nsec / 1e9
> +    end,
> +
> +    minutes = function(self)
> +        return (tonumber(self.secs) + self.nsec / 1e9) / 60
> +    end,
> +
> +    hours = function(self)
> +        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60)
> +    end,
> +
> +    days = function(self)
> +        return (tonumber(self.secs) + self.nsec / 1e9) / (60 * 60) / 24
> +    end,
> +}
> +
> +local datetime_index = function(self, key)
> +    return datetime_index_handlers[key] ~= nil and
> +           datetime_index_handlers[key](self) or nil
> +end
> +
> +local datetime_newindex_handlers = {
> +    unixtime = function(self, value)
> +        self.secs = value
> +        self.nsec, self.offset = 0, 0
> +    end,
> +
> +    timestamp = function(self, value)
> +        local secs, frac = math.modf(value)
> +        self.secs = secs
> +        self.nsec = frac * 1e9
> +        self.offset = 0
> +    end,
> +}
> +
> +local function datetime_newindex(self, key, value)
> +    if datetime_newindex_handlers[key] ~= nil then
> +        datetime_newindex_handlers[key](self, value)
> +    end
> +end
> +
> +local function datetime_new_raw(secs, nsec, offset)
> +    local dt_obj = ffi.new(datetime_t)
> +    dt_obj.secs = secs
> +    dt_obj.nsec = nsec
> +    dt_obj.offset = offset
> +    return dt_obj
> +end
> +
> +local function mk_timestamp(dt, sp, fp, offset)
> +    local epochV = dt ~= nil and (builtin.dt_rdn(dt) - DT_EPOCH_1970_OFFSET) *
> +                   SECS_PER_DAY or 0
> +    local spV = sp ~= nil and sp or 0
> +    local fpV = fp ~= nil and fp or 0
> +    local ofsV = offset ~= nil and offset or 0
> +    return datetime_new_raw (epochV + spV - ofsV * 60, fpV, ofsV)
> +end
> +
> +-- create @datetime_t given object @o fields
> +local function datetime_new(o)
> +    if o == nil then
> +        return datetime_new_raw(0, 0, 0)
> +    end
> +    local secs = 0
> +    local nsec = 0
> +    local offset = 0
> +    local easy_way = false
> +    local y = 0
> +    local M = 0
> +    local d = 0
> +    local ymd = false
> +
> +    local h = 0
> +    local m = 0
> +    local s = 0
> +    local frac = 0
> +    local hms = false
> +
> +    local dt = 0
> +
> +    local handlers = {
> +        secs = function(_, v)
> +            secs = v
> +            easy_way = true
> +        end,
> +
> +        nsec = function(_, v)
> +            nsec = v
> +            easy_way = true
> +        end,
> +
> +        offset = function (_, v)
> +            offset = v
> +            easy_way = true
> +        end,
> +
> +        year = function(k, v)
> +            check_range(v, {1, 9999}, k)
> +            y = v
> +            ymd = true
> +        end,
> +
> +        month = function(k, v)
> +            check_range(v, {1, 12}, k)
> +            M = v
> +            ymd = true
> +        end,
> +
> +        day = function(k, v)
> +            check_range(v, {1, 31}, k)
> +            d = v
> +            ymd = true
> +        end,
> +
> +        hour = function(k, v)
> +            check_range(v, {0, 23}, k)
> +            h = v
> +            hms = true
> +        end,
> +
> +        minute = function(k, v)
> +            check_range(v, {0, 59}, k)
> +            m = v
> +            hms = true
> +        end,
> +
> +        second = function(k, v)
> +            check_range(v, {0, 60}, k)
> +            s, frac = math.modf(v)
> +            frac = frac * 1e9 -- convert fraction to nanoseconds
> +            hms = true
> +        end,
> +
> +        -- tz offset in minutes
> +        tz = function(k, v)
> +            check_range(v, {0, 720}, k)
> +            offset = v
> +        end
> +    }
> +    for key, value in pairs(o) do
> +        handlers[key](key, value)
> +    end
> +
> +    -- .sec, .nsec, .offset
> +    if easy_way then
> +        return datetime_new_raw(secs, nsec, offset)
> +    end
> +
> +    -- .year, .month, .day
> +    if ymd then
> +        dt = dt + builtin.dt_from_ymd(y, M, d)
> +    end
> +
> +    -- .hour, .minute, .second
> +    if hms then
> +        secs = h * 3600 + m * 60 + s
> +    end
> +
> +    return mk_timestamp(dt, secs, frac, offset)
> +end
> +
> +local function date_first(lhs, rhs)
> +    if is_datetime(lhs) then
> +        return lhs, rhs
> +    else
> +        return rhs, lhs
> +    end
> +end
> +
> +local function error_incompatible(name)
> +    error(("datetime:%s() - incompatible type of arguments"):
> +          format(name), 3)
> +end
> +
> +local function datetime_sub(lhs, rhs)
> +    check_date_interval(lhs, "operator -")
> +    local d, s = lhs, rhs
> +    local left_t = ffi.typeof(d)
> +    local right_t = ffi.typeof(s)
> +    local o
> +
> +    if left_t == datetime_t then
> +        -- left is date, right is date or generic interval
> +        if (right_t == datetime_t or right_t == interval_t) then
> +            o = right_t == datetime_t and interval_new() or datetime_new()
> +            o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
> +                                            lhs.nsec - rhs.nsec)
> +            return o
> +        else
> +            error_incompatible("operator -")
> +        end
> +    -- both left and right are generic intervals
> +    elseif left_t == interval_t and right_t == interval_t then
> +        o = interval_new()
> +        o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
> +                                        lhs.nsec - rhs.nsec)
> +        return o
> +    else
> +        error_incompatible("operator -")
> +    end
> +end
> +
> +local function datetime_add(lhs, rhs)
> +    local d, s = date_first(lhs, rhs)
> +
> +    check_date_interval(d, "operator +")
> +    check_interval(s, "operator +")
> +    local left_t = ffi.typeof(d)
> +    local right_t = ffi.typeof(s)
> +    local o
> +
> +    -- left is date, right is date or interval
> +    if left_t == datetime_t and right_t == interval_t then
> +        o = datetime_new()
> +        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
> +        return o
> +    -- both left and right are generic intervals
> +    elseif left_t == interval_t and right_t == interval_t then
> +        o = interval_new()
> +        o.secs, o.nsec = _normalize_nsec(d.secs + s.secs, d.nsec + s.nsec)
> +        return o
> +    else
> +        error_incompatible("operator +")
> +    end
> +end
> +
> +-- simple parse functions:
> +-- parse_date/parse_time/parse_zone
> +
> +--[[
> +    Basic      Extended
> +    20121224   2012-12-24   Calendar date   (ISO 8601)
> +    2012359    2012-359     Ordinal date    (ISO 8601)
> +    2012W521   2012-W52-1   Week date       (ISO 8601)
> +    2012Q485   2012-Q4-85   Quarter date
> +]]
> +
> +local function parse_date(str)
> +    check_str("datetime.parse_date()")
> +    local dt = ffi.new('dt_t[1]')
> +    local len = builtin.dt_parse_iso_date(str, #str, dt)
> +    return len > 0 and mk_timestamp(dt[0]) or nil, tonumber(len)
> +end
> +
> +--[[
> +    Basic               Extended
> +    T12                 N/A
> +    T1230               T12:30
> +    T123045             T12:30:45
> +    T123045.123456789   T12:30:45.123456789
> +    T123045,123456789   T12:30:45,123456789
> +
> +    The time designator [T] may be omitted.
> +]]
> +local function parse_time(str)
> +    check_str("datetime.parse_time()")
> +    local sp = ffi.new('int[1]')
> +    local fp = ffi.new('int[1]')
> +    local len = builtin.dt_parse_iso_time(str, #str, sp, fp)
> +    return len > 0 and mk_timestamp(nil, sp[0], fp[0]) or nil,
> +           tonumber(len)
> +end
> +
> +--[[
> +    Basic    Extended
> +    Z        N/A
> +    +hh      N/A
> +    -hh      N/A
> +    +hhmm    +hh:mm
> +    -hhmm    -hh:mm
> +]]
> +local function parse_zone(str)
> +    check_str("datetime.parse_zone()")
> +    local offset = ffi.new('int[1]')
> +    local len = builtin.dt_parse_iso_zone_lenient(str, #str, offset)
> +    return len > 0 and mk_timestamp(nil, nil, nil, offset[0]) or nil,
> +           tonumber(len)
> +end
> +
> +
> +--[[
> +    aggregated parse functions
> +    assumes to deal with date T time time_zone
> +    at once
> +
> +    date [T] time [ ] time_zone
> +]]
> +local function parse(str)
> +    check_str("datetime.parse()")
> +    local dt = ffi.new('dt_t[1]')
> +    local len = #str
> +    local n = builtin.dt_parse_iso_date(str, len, dt)
> +    local dt_ = dt[0]
> +    if n == 0 or len == n then
> +        return mk_timestamp(dt_)
> +    end
> +
> +    str = str:sub(tonumber(n) + 1)
> +
> +    local ch = str:sub(1,1)
> +    if ch:match('[Tt ]') == nil then
> +        return mk_timestamp(dt_)
> +    end
> +
> +    str = str:sub(2)
> +    len = #str
> +
> +    local sp = ffi.new('int[1]')
> +    local fp = ffi.new('int[1]')
> +    local n = builtin.dt_parse_iso_time(str, len, sp, fp)
> +    if n == 0 then
> +        return mk_timestamp(dt_)
> +    end
> +    local sp_ = sp[0]
> +    local fp_ = fp[0]
> +    if len == n then
> +        return mk_timestamp(dt_, sp_, fp_)
> +    end
> +
> +    str = str:sub(tonumber(n) + 1)
> +
> +    if str:sub(1,1) == ' ' then
> +        str = str:sub(2)
> +    end
> +
> +    len = #str
> +
> +    local offset = ffi.new('int[1]')
> +    n = builtin.dt_parse_iso_zone_lenient(str, len, offset)
> +    if n == 0 then
> +        return mk_timestamp(dt_, sp_, fp_)
> +    end
> +    return mk_timestamp(dt_, sp_, fp_, offset[0])
> +end
> +
> +local function datetime_from(o)
> +    if o == nil or type(o) == 'table' then
> +        return datetime_new(o)
> +    elseif type(o) == 'string' then
> +        return parse(o)
> +    end
> +end
> +
> +local function local_now()
> +    local p_tv = ffi.new('struct timeval [1]')
> +    local rc = builtin.gettimeofday(p_tv, nil)
> +    assert(rc == 0)
> +
> +    local secs = p_tv[0].tv_sec
> +    local nsec = p_tv[0].tv_usec * 1000
> +
> +    local p_time = ffi.new('time_t[1]')
> +    local p_tm = ffi.new('struct tm[1]')
> +    builtin.time(p_time)
> +    builtin.localtime_r(p_time, p_tm)
> +    local ofs = p_tm[0].tm_gmtoff / 60 -- convert seconds to minutes
> +
> +    return datetime_new_raw(secs, nsec, ofs)
> +end
> +
> +local function datetime_to_tm_ptr(o)
> +    assert(is_datetime(o))
> +    local p_tm = ffi.new('struct tm[1]')
> +    -- dt_to_struct_tm() fills only date data
> +    builtin.dt_to_struct_tm(local_dt(o), p_tm)
> +
> +    -- calculate the smaller data (hour, minute,
> +    -- seconds) using datetime seconds value
> +    local seconds_of_day = o.secs % 86400
> +    local hour = (seconds_of_day / 3600) % 24
> +    local minute = (seconds_of_day / 60) % 60
> +    p_tm[0].tm_sec = seconds_of_day % 60
> +    p_tm[0].tm_min = minute
> +    p_tm[0].tm_hour = hour
> +
> +    p_tm[0].tm_gmtoff = o.offset * 60
> +
> +    return p_tm
> +end
> +
> +local function asctime(o)
> +    check_date(o, "datetime:asctime()")
> +
> +    local p_tm = datetime_to_tm_ptr(o)
> +    return ffi.string(builtin.asctime(p_tm))
> +end
> +
> +local function ctime(o)
> +    check_date(o, "datetime:ctime()")
> +    local p_time = ffi.new('time_t[1]')
> +    p_time[0] = o.secs
> +    return ffi.string(builtin.ctime(p_time))
> +end
> +
> +local function strftime(fmt, o)
> +    check_date(o, "datetime.strftime()")
> +    local p_tm = datetime_to_tm_ptr(o)
> +    local sz = builtin.strftime(nil, 1024, fmt, p_tm) + 1
> +    local buff = ffi.new('char[?]', sz)
> +    builtin.strftime(buff, sz, fmt, p_tm)
> +    return ffi.string(buff)
> +end
> +
> +local datetime_mt = {
> +    __serialize = datetime_serialize,
> +    __eq = datetime_eq,
> +    __lt = datetime_lt,
> +    __le = datetime_le,
> +    __sub = datetime_sub,
> +    __add = datetime_add,
> +    __index = datetime_index,
> +    __newindex = datetime_newindex,
> +}
> +
> +local interval_mt = {
> +    __serialize = interval_serialize,
> +    __eq = datetime_eq,
> +    __lt = datetime_lt,
> +    __le = datetime_le,
> +    __sub = datetime_sub,
> +    __add = datetime_add,
> +    __index = datetime_index,
> +}
> +
> +ffi.metatype(interval_t, interval_mt)
> +ffi.metatype(datetime_t, datetime_mt)
> +
> +return setmetatable(
> +    {
> +        new         = datetime_new,
> +        interval    = interval_new,
> +
> +        parse       = parse,
> +        parse_date  = parse_date,
> +        parse_time  = parse_time,
> +        parse_zone  = parse_zone,
> +
> +        now         = local_now,
> +        strftime    = strftime,
> +        asctime     = asctime,
> +        ctime       = ctime,
> +
> +        is_datetime = is_datetime,
> +        is_interval = is_interval,
> +    }, {
> +        __call = function(self, ...) return datetime_from(...) end
> +    }
> +)
> diff --git a/src/lua/init.c b/src/lua/init.c
> index f9738025d..127e935d7 100644
> --- a/src/lua/init.c
> +++ b/src/lua/init.c
> @@ -129,7 +129,8 @@ extern char strict_lua[],
>   	parse_lua[],
>   	process_lua[],
>   	humanize_lua[],
> -	memprof_lua[]
> +	memprof_lua[],
> +	datetime_lua[]
>   ;
>   
>   static const char *lua_modules[] = {
> @@ -184,6 +185,7 @@ static const char *lua_modules[] = {
>   	"memprof.process", process_lua,
>   	"memprof.humanize", humanize_lua,
>   	"memprof", memprof_lua,
> +	"datetime", datetime_lua,
>   	NULL
>   };
>   
> diff --git a/src/lua/utils.c b/src/lua/utils.c
> index ebc19206c..611044b6f 100644
> --- a/src/lua/utils.c
> +++ b/src/lua/utils.c
> @@ -47,6 +47,9 @@ static uint32_t CTID_STRUCT_IBUF_PTR;
>   uint32_t CTID_CHAR_PTR;
>   uint32_t CTID_CONST_CHAR_PTR;
>   uint32_t CTID_UUID;
> +uint32_t CTID_DATETIME = 0;
> +uint32_t CTID_INTERVAL = 0;
> +
>   
>   void *
>   luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
> @@ -107,6 +110,12 @@ luaL_pushuuid(struct lua_State *L)
>   	return luaL_pushcdata(L, CTID_UUID);
>   }
>   
> +struct datetime_t *
> +luaL_pushdatetime(struct lua_State *L)
> +{
> +	return luaL_pushcdata(L, CTID_DATETIME);
> +}
> +
>   int
>   luaL_iscdata(struct lua_State *L, int idx)
>   {
> @@ -712,6 +721,24 @@ tarantool_lua_utils_init(struct lua_State *L)
>   	CTID_UUID = luaL_ctypeid(L, "struct tt_uuid");
>   	assert(CTID_UUID != 0);
>   
> +	rc = luaL_cdef(L, "struct datetime_t {"
> +			  "int64_t secs;"
> +			  "int32_t nsec;"
> +			  "int32_t offset;"
> +			  "};");
> +	assert(rc == 0);
> +	(void) rc;
> +	CTID_DATETIME = luaL_ctypeid(L, "struct datetime_t");
> +	assert(CTID_DATETIME != 0);
> +	rc = luaL_cdef(L, "struct datetime_interval_t {"
> +			  "int64_t secs;"
> +			  "int32_t nsec;"
> +			  "};");
> +	assert(rc == 0);
> +	(void) rc;
> +	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval_t");
> +	assert(CTID_INTERVAL != 0);
> +
>   	lua_pushcfunction(L, luaT_newthread_wrapper);
>   	luaT_newthread_ref = luaL_ref(L, LUA_REGISTRYINDEX);
>   	return 0;
> diff --git a/src/lua/utils.h b/src/lua/utils.h
> index 969edca45..bf56b1bb5 100644
> --- a/src/lua/utils.h
> +++ b/src/lua/utils.h
> @@ -59,6 +59,7 @@ struct lua_State;
>   struct ibuf;
>   typedef struct ibuf box_ibuf_t;
>   struct tt_uuid;
> +struct datetime_t;
>   
>   /**
>    * Single global lua_State shared by core and modules.
> @@ -71,10 +72,21 @@ extern struct lua_State *tarantool_L;
>   extern uint32_t CTID_CHAR_PTR;
>   extern uint32_t CTID_CONST_CHAR_PTR;
>   extern uint32_t CTID_UUID;
> +extern uint32_t CTID_DATETIME;
> +extern uint32_t CTID_INTERVAL;
>   
>   struct tt_uuid *
>   luaL_pushuuid(struct lua_State *L);
>   
> +/**
> + * @brief Push cdata of a datetime_t type onto the stack.
> + * @param L Lua State
> + * @sa luaL_pushcdata
> + * @return memory associated with this datetime_t data
> + */
> +struct datetime_t *
> +luaL_pushdatetime(struct lua_State *L);
> +
>   /** \cond public */
>   
>   /**
> diff --git a/test/unit/CMakeLists.txt b/test/unit/CMakeLists.txt
> index 5bb7cd6e7..f8320aebd 100644
> --- a/test/unit/CMakeLists.txt
> +++ b/test/unit/CMakeLists.txt
> @@ -56,6 +56,8 @@ add_executable(uuid.test uuid.c core_test_utils.c)
>   target_link_libraries(uuid.test uuid unit)
>   add_executable(random.test random.c core_test_utils.c)
>   target_link_libraries(random.test core unit)
> +add_executable(datetime.test datetime.c)
> +target_link_libraries(datetime.test cdt unit)
>   
>   add_executable(bps_tree.test bps_tree.cc)
>   target_link_libraries(bps_tree.test small misc)
> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
> new file mode 100644
> index 000000000..b6f568c03
> --- /dev/null
> +++ b/test/unit/datetime.c
> @@ -0,0 +1,221 @@
> +#include "dt.h"
> +#include <assert.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <time.h>
> +
> +#include "unit.h"
> +
> +const char sample[] = "2012-12-24T15:30Z";
> +
> +#define S(s) {s, sizeof(s) - 1}
> +struct {
> +	const char * sz;
> +	size_t len;
> +} tests[] = {
> +	S("2012-12-24 15:30Z"),
> +	S("2012-12-24 15:30z"),
> +	S("2012-12-24 15:30"),
> +	S("2012-12-24 16:30+01:00"),
> +	S("2012-12-24 16:30+0100"),
> +	S("2012-12-24 16:30+01"),
> +	S("2012-12-24 14:30-01:00"),
> +	S("2012-12-24 14:30-0100"),
> +	S("2012-12-24 14:30-01"),
> +	S("2012-12-24 15:30:00Z"),
> +	S("2012-12-24 15:30:00z"),
> +	S("2012-12-24 15:30:00"),
> +	S("2012-12-24 16:30:00+01:00"),
> +	S("2012-12-24 16:30:00+0100"),
> +	S("2012-12-24 14:30:00-01:00"),
> +	S("2012-12-24 14:30:00-0100"),
> +	S("2012-12-24 15:30:00.123456Z"),
> +	S("2012-12-24 15:30:00.123456z"),
> +	S("2012-12-24 15:30:00.123456"),
> +	S("2012-12-24 16:30:00.123456+01:00"),
> +	S("2012-12-24 16:30:00.123456+01"),
> +	S("2012-12-24 14:30:00.123456-01:00"),
> +	S("2012-12-24 14:30:00.123456-01"),
> +	S("2012-12-24t15:30Z"),
> +	S("2012-12-24t15:30z"),
> +	S("2012-12-24t15:30"),
> +	S("2012-12-24t16:30+01:00"),
> +	S("2012-12-24t16:30+0100"),
> +	S("2012-12-24t14:30-01:00"),
> +	S("2012-12-24t14:30-0100"),
> +	S("2012-12-24t15:30:00Z"),
> +	S("2012-12-24t15:30:00z"),
> +	S("2012-12-24t15:30:00"),
> +	S("2012-12-24t16:30:00+01:00"),
> +	S("2012-12-24t16:30:00+0100"),
> +	S("2012-12-24t14:30:00-01:00"),
> +	S("2012-12-24t14:30:00-0100"),
> +	S("2012-12-24t15:30:00.123456Z"),
> +	S("2012-12-24t15:30:00.123456z"),
> +	S("2012-12-24t16:30:00.123456+01:00"),
> +	S("2012-12-24t14:30:00.123456-01:00"),
> +	S("2012-12-24 16:30 +01:00"),
> +	S("2012-12-24 14:30 -01:00"),
> +	S("2012-12-24 15:30 UTC"),
> +	S("2012-12-24 16:30 UTC+1"),
> +	S("2012-12-24 16:30 UTC+01"),
> +	S("2012-12-24 16:30 UTC+0100"),
> +	S("2012-12-24 16:30 UTC+01:00"),
> +	S("2012-12-24 14:30 UTC-1"),
> +	S("2012-12-24 14:30 UTC-01"),
> +	S("2012-12-24 14:30 UTC-01:00"),
> +	S("2012-12-24 14:30 UTC-0100"),
> +	S("2012-12-24 15:30 GMT"),
> +	S("2012-12-24 16:30 GMT+1"),
> +	S("2012-12-24 16:30 GMT+01"),
> +	S("2012-12-24 16:30 GMT+0100"),
> +	S("2012-12-24 16:30 GMT+01:00"),
> +	S("2012-12-24 14:30 GMT-1"),
> +	S("2012-12-24 14:30 GMT-01"),
> +	S("2012-12-24 14:30 GMT-01:00"),
> +	S("2012-12-24 14:30 GMT-0100"),
> +	S("2012-12-24 14:30 -01:00"),
> +	S("2012-12-24 16:30:00 +01:00"),
> +	S("2012-12-24 14:30:00 -01:00"),
> +	S("2012-12-24 16:30:00.123456 +01:00"),
> +	S("2012-12-24 14:30:00.123456 -01:00"),
> +	S("2012-12-24 15:30:00.123456 -00:00"),
> +	S("20121224T1630+01:00"),
> +	S("2012-12-24T1630+01:00"),
> +	S("20121224T16:30+01"),
> +	S("20121224T16:30 +01"),
> +};
> +#undef S
> +
> +#define DIM(a) (sizeof(a) / sizeof(a[0]))
> +
> +// p5-time-moment/src/moment_parse.c: parse_string_lenient()
> +static int
> +parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
> +	       int32_t *op)
> +{
> +	size_t n;
> +	dt_t dt;
> +	char c;
> +	int sod = 0, nanosecond = 0, offset = 0;
> +
> +	n = dt_parse_iso_date(str, len, &dt);
> +	if (!n)
> +		return 1;
> +	if (n == len)
> +		goto exit;
> +
> +	c = str[n++];
> +	if (!(c == 'T' || c == 't' || c == ' '))
> +		return 1;
> +
> +	str += n;
> +	len -= n;
> +
> +	n = dt_parse_iso_time(str, len, &sod, &nanosecond);
> +	if (!n)
> +		return 1;
> +	if (n == len)
> +		goto exit;
> +
> +	if (str[n] == ' ')
> +	n++;
> +
> +	str += n;
> +	len -= n;
> +
> +	n = dt_parse_iso_zone_lenient(str, len, &offset);
> +	if (!n || n != len)
> +		return 1;
> +
> +exit:
> +	*sp = ((int64_t)dt_rdn(dt) - 719163) * 86400 + sod - offset * 60;
> +	*np = nanosecond;
> +	*op = offset;
> +
> +	return 0;
> +}
> +
> +// avoid introducing external datetime.h dependency
> +// - just copy paste it for today
> +#define SECS_PER_DAY      86400
> +#define NANOS_PER_SEC     1000000000
> +#define DT_EPOCH_1970_OFFSET 719163
> +
> +
> +struct datetime_t {
> +	int64_t secs;
> +	int32_t nsec;
> +	int32_t offset;
> +};
> +
> +static int
> +local_rd(const struct datetime_t * dt) {
> +	return (int)(dt->secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET;
> +}
> +
> +static int
> +local_dt(const struct datetime_t * dt) {
> +	return dt_from_rdn(local_rd(dt));
> +}
> +
> +
> +struct tm*
> +datetime_to_tm(struct datetime_t * dt)
> +{
> +	static struct tm tm;
> +
> +	memset(&tm, 0, sizeof(tm));
> +	dt_to_struct_tm(local_dt(dt), &tm);
> +
> +	int seconds_of_day = dt->secs % 86400;
> +	tm.tm_hour = (seconds_of_day / 3600) % 24;
> +	tm.tm_min = (seconds_of_day / 60) % 60;
> +	tm.tm_sec = seconds_of_day % 60;
> +
> +	return &tm;
> +}
> +
> +static void datetime_test(void)
> +{
> +	size_t index;
> +	int64_t secs_expected;
> +	int32_t nanosecs;
> +	int32_t ofs;
> +
> +	plan(355);
> +	parse_datetime(sample, sizeof(sample) - 1,
> +		       &secs_expected, &nanosecs, &ofs);
> +
> +	for (index = 0; index < DIM(tests); index++) {
> +		int64_t secs;
> +		int rc = parse_datetime(tests[index].sz, tests[index].len,
> +						&secs, &nanosecs, &ofs);
> +		is(rc, 0, "correct parse_datetime return value for '%s'",
> +		   tests[index].sz);
> +		is(secs, secs_expected, "correct parse_datetime output "
> +		   "seconds for '%s", tests[index].sz);
> +
> +		// check that stringized literal produces the same date
> +		// time fields
> +		static char buff[40];
> +		struct datetime_t dt = {secs, nanosecs, ofs};
> +		// datetime_to_tm returns time in GMT zone
> +		struct tm * p_tm = datetime_to_tm(&dt);
> +		size_t len = strftime(buff, sizeof buff, "%F %T%z", p_tm);
> +		ok(len > 0, "strftime");
> +		rc = parse_datetime(buff, len, &dt.secs, &dt.nsec, &dt.offset);
> +		is(rc, 0, "correct parse_datetime return value for '%s'", buff);
> +		is(secs, dt.secs,
> +		   "reversible seconds via strftime for '%s", buff);
> +	}
> +}
> +
> +int
> +main(void)
> +{
> +	plan(1);
> +	datetime_test();
> +
> +	return check_plan();
> +}
> diff --git a/test/unit/datetime.result b/test/unit/datetime.result
> new file mode 100644
> index 000000000..f106fa769
> --- /dev/null
> +++ b/test/unit/datetime.result
> @@ -0,0 +1,358 @@
> +1..1
> +    1..355
> +    ok 1 - correct parse_datetime return value for '2012-12-24 15:30Z'
> +    ok 2 - correct parse_datetime output seconds for '2012-12-24 15:30Z
> +    ok 3 - strftime
> +    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 6 - correct parse_datetime return value for '2012-12-24 15:30z'
> +    ok 7 - correct parse_datetime output seconds for '2012-12-24 15:30z
> +    ok 8 - strftime
> +    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 11 - correct parse_datetime return value for '2012-12-24 15:30'
> +    ok 12 - correct parse_datetime output seconds for '2012-12-24 15:30
> +    ok 13 - strftime
> +    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 16 - correct parse_datetime return value for '2012-12-24 16:30+01:00'
> +    ok 17 - correct parse_datetime output seconds for '2012-12-24 16:30+01:00
> +    ok 18 - strftime
> +    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 21 - correct parse_datetime return value for '2012-12-24 16:30+0100'
> +    ok 22 - correct parse_datetime output seconds for '2012-12-24 16:30+0100
> +    ok 23 - strftime
> +    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 26 - correct parse_datetime return value for '2012-12-24 16:30+01'
> +    ok 27 - correct parse_datetime output seconds for '2012-12-24 16:30+01
> +    ok 28 - strftime
> +    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 31 - correct parse_datetime return value for '2012-12-24 14:30-01:00'
> +    ok 32 - correct parse_datetime output seconds for '2012-12-24 14:30-01:00
> +    ok 33 - strftime
> +    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 36 - correct parse_datetime return value for '2012-12-24 14:30-0100'
> +    ok 37 - correct parse_datetime output seconds for '2012-12-24 14:30-0100
> +    ok 38 - strftime
> +    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 41 - correct parse_datetime return value for '2012-12-24 14:30-01'
> +    ok 42 - correct parse_datetime output seconds for '2012-12-24 14:30-01
> +    ok 43 - strftime
> +    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 46 - correct parse_datetime return value for '2012-12-24 15:30:00Z'
> +    ok 47 - correct parse_datetime output seconds for '2012-12-24 15:30:00Z
> +    ok 48 - strftime
> +    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 51 - correct parse_datetime return value for '2012-12-24 15:30:00z'
> +    ok 52 - correct parse_datetime output seconds for '2012-12-24 15:30:00z
> +    ok 53 - strftime
> +    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 56 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 57 - correct parse_datetime output seconds for '2012-12-24 15:30:00
> +    ok 58 - strftime
> +    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 61 - correct parse_datetime return value for '2012-12-24 16:30:00+01:00'
> +    ok 62 - correct parse_datetime output seconds for '2012-12-24 16:30:00+01:00
> +    ok 63 - strftime
> +    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 66 - correct parse_datetime return value for '2012-12-24 16:30:00+0100'
> +    ok 67 - correct parse_datetime output seconds for '2012-12-24 16:30:00+0100
> +    ok 68 - strftime
> +    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 71 - correct parse_datetime return value for '2012-12-24 14:30:00-01:00'
> +    ok 72 - correct parse_datetime output seconds for '2012-12-24 14:30:00-01:00
> +    ok 73 - strftime
> +    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 76 - correct parse_datetime return value for '2012-12-24 14:30:00-0100'
> +    ok 77 - correct parse_datetime output seconds for '2012-12-24 14:30:00-0100
> +    ok 78 - strftime
> +    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 81 - correct parse_datetime return value for '2012-12-24 15:30:00.123456Z'
> +    ok 82 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456Z
> +    ok 83 - strftime
> +    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 86 - correct parse_datetime return value for '2012-12-24 15:30:00.123456z'
> +    ok 87 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456z
> +    ok 88 - strftime
> +    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 91 - correct parse_datetime return value for '2012-12-24 15:30:00.123456'
> +    ok 92 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456
> +    ok 93 - strftime
> +    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 96 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01:00'
> +    ok 97 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01:00
> +    ok 98 - strftime
> +    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 101 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01'
> +    ok 102 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01
> +    ok 103 - strftime
> +    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 106 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01:00'
> +    ok 107 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01:00
> +    ok 108 - strftime
> +    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 111 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01'
> +    ok 112 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01
> +    ok 113 - strftime
> +    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 116 - correct parse_datetime return value for '2012-12-24t15:30Z'
> +    ok 117 - correct parse_datetime output seconds for '2012-12-24t15:30Z
> +    ok 118 - strftime
> +    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 121 - correct parse_datetime return value for '2012-12-24t15:30z'
> +    ok 122 - correct parse_datetime output seconds for '2012-12-24t15:30z
> +    ok 123 - strftime
> +    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 126 - correct parse_datetime return value for '2012-12-24t15:30'
> +    ok 127 - correct parse_datetime output seconds for '2012-12-24t15:30
> +    ok 128 - strftime
> +    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 131 - correct parse_datetime return value for '2012-12-24t16:30+01:00'
> +    ok 132 - correct parse_datetime output seconds for '2012-12-24t16:30+01:00
> +    ok 133 - strftime
> +    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 136 - correct parse_datetime return value for '2012-12-24t16:30+0100'
> +    ok 137 - correct parse_datetime output seconds for '2012-12-24t16:30+0100
> +    ok 138 - strftime
> +    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 141 - correct parse_datetime return value for '2012-12-24t14:30-01:00'
> +    ok 142 - correct parse_datetime output seconds for '2012-12-24t14:30-01:00
> +    ok 143 - strftime
> +    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 146 - correct parse_datetime return value for '2012-12-24t14:30-0100'
> +    ok 147 - correct parse_datetime output seconds for '2012-12-24t14:30-0100
> +    ok 148 - strftime
> +    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 151 - correct parse_datetime return value for '2012-12-24t15:30:00Z'
> +    ok 152 - correct parse_datetime output seconds for '2012-12-24t15:30:00Z
> +    ok 153 - strftime
> +    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 156 - correct parse_datetime return value for '2012-12-24t15:30:00z'
> +    ok 157 - correct parse_datetime output seconds for '2012-12-24t15:30:00z
> +    ok 158 - strftime
> +    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 161 - correct parse_datetime return value for '2012-12-24t15:30:00'
> +    ok 162 - correct parse_datetime output seconds for '2012-12-24t15:30:00
> +    ok 163 - strftime
> +    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 166 - correct parse_datetime return value for '2012-12-24t16:30:00+01:00'
> +    ok 167 - correct parse_datetime output seconds for '2012-12-24t16:30:00+01:00
> +    ok 168 - strftime
> +    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 171 - correct parse_datetime return value for '2012-12-24t16:30:00+0100'
> +    ok 172 - correct parse_datetime output seconds for '2012-12-24t16:30:00+0100
> +    ok 173 - strftime
> +    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 176 - correct parse_datetime return value for '2012-12-24t14:30:00-01:00'
> +    ok 177 - correct parse_datetime output seconds for '2012-12-24t14:30:00-01:00
> +    ok 178 - strftime
> +    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 181 - correct parse_datetime return value for '2012-12-24t14:30:00-0100'
> +    ok 182 - correct parse_datetime output seconds for '2012-12-24t14:30:00-0100
> +    ok 183 - strftime
> +    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 186 - correct parse_datetime return value for '2012-12-24t15:30:00.123456Z'
> +    ok 187 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456Z
> +    ok 188 - strftime
> +    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 191 - correct parse_datetime return value for '2012-12-24t15:30:00.123456z'
> +    ok 192 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456z
> +    ok 193 - strftime
> +    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 196 - correct parse_datetime return value for '2012-12-24t16:30:00.123456+01:00'
> +    ok 197 - correct parse_datetime output seconds for '2012-12-24t16:30:00.123456+01:00
> +    ok 198 - strftime
> +    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 201 - correct parse_datetime return value for '2012-12-24t14:30:00.123456-01:00'
> +    ok 202 - correct parse_datetime output seconds for '2012-12-24t14:30:00.123456-01:00
> +    ok 203 - strftime
> +    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 206 - correct parse_datetime return value for '2012-12-24 16:30 +01:00'
> +    ok 207 - correct parse_datetime output seconds for '2012-12-24 16:30 +01:00
> +    ok 208 - strftime
> +    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 211 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
> +    ok 212 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
> +    ok 213 - strftime
> +    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 216 - correct parse_datetime return value for '2012-12-24 15:30 UTC'
> +    ok 217 - correct parse_datetime output seconds for '2012-12-24 15:30 UTC
> +    ok 218 - strftime
> +    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 221 - correct parse_datetime return value for '2012-12-24 16:30 UTC+1'
> +    ok 222 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+1
> +    ok 223 - strftime
> +    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 226 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01'
> +    ok 227 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01
> +    ok 228 - strftime
> +    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 231 - correct parse_datetime return value for '2012-12-24 16:30 UTC+0100'
> +    ok 232 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+0100
> +    ok 233 - strftime
> +    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 236 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01:00'
> +    ok 237 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01:00
> +    ok 238 - strftime
> +    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 241 - correct parse_datetime return value for '2012-12-24 14:30 UTC-1'
> +    ok 242 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-1
> +    ok 243 - strftime
> +    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 246 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01'
> +    ok 247 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01
> +    ok 248 - strftime
> +    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 251 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01:00'
> +    ok 252 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01:00
> +    ok 253 - strftime
> +    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 256 - correct parse_datetime return value for '2012-12-24 14:30 UTC-0100'
> +    ok 257 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-0100
> +    ok 258 - strftime
> +    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 261 - correct parse_datetime return value for '2012-12-24 15:30 GMT'
> +    ok 262 - correct parse_datetime output seconds for '2012-12-24 15:30 GMT
> +    ok 263 - strftime
> +    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 266 - correct parse_datetime return value for '2012-12-24 16:30 GMT+1'
> +    ok 267 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+1
> +    ok 268 - strftime
> +    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 271 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01'
> +    ok 272 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01
> +    ok 273 - strftime
> +    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 276 - correct parse_datetime return value for '2012-12-24 16:30 GMT+0100'
> +    ok 277 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+0100
> +    ok 278 - strftime
> +    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 281 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01:00'
> +    ok 282 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01:00
> +    ok 283 - strftime
> +    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 286 - correct parse_datetime return value for '2012-12-24 14:30 GMT-1'
> +    ok 287 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-1
> +    ok 288 - strftime
> +    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 291 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01'
> +    ok 292 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01
> +    ok 293 - strftime
> +    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 296 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01:00'
> +    ok 297 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01:00
> +    ok 298 - strftime
> +    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 301 - correct parse_datetime return value for '2012-12-24 14:30 GMT-0100'
> +    ok 302 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-0100
> +    ok 303 - strftime
> +    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 306 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
> +    ok 307 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
> +    ok 308 - strftime
> +    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 311 - correct parse_datetime return value for '2012-12-24 16:30:00 +01:00'
> +    ok 312 - correct parse_datetime output seconds for '2012-12-24 16:30:00 +01:00
> +    ok 313 - strftime
> +    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 316 - correct parse_datetime return value for '2012-12-24 14:30:00 -01:00'
> +    ok 317 - correct parse_datetime output seconds for '2012-12-24 14:30:00 -01:00
> +    ok 318 - strftime
> +    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 321 - correct parse_datetime return value for '2012-12-24 16:30:00.123456 +01:00'
> +    ok 322 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456 +01:00
> +    ok 323 - strftime
> +    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 326 - correct parse_datetime return value for '2012-12-24 14:30:00.123456 -01:00'
> +    ok 327 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456 -01:00
> +    ok 328 - strftime
> +    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 331 - correct parse_datetime return value for '2012-12-24 15:30:00.123456 -00:00'
> +    ok 332 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456 -00:00
> +    ok 333 - strftime
> +    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 336 - correct parse_datetime return value for '20121224T1630+01:00'
> +    ok 337 - correct parse_datetime output seconds for '20121224T1630+01:00
> +    ok 338 - strftime
> +    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 341 - correct parse_datetime return value for '2012-12-24T1630+01:00'
> +    ok 342 - correct parse_datetime output seconds for '2012-12-24T1630+01:00
> +    ok 343 - strftime
> +    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 346 - correct parse_datetime return value for '20121224T16:30+01'
> +    ok 347 - correct parse_datetime output seconds for '20121224T16:30+01
> +    ok 348 - strftime
> +    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 351 - correct parse_datetime return value for '20121224T16:30 +01'
> +    ok 352 - correct parse_datetime output seconds for '20121224T16:30 +01
> +    ok 353 - strftime
> +    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> +    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +ok 1 - subtests
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 9/9] lua, box, datetime: rename struct datetime_t
  2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 9/9] lua, box, datetime: rename struct datetime_t Timur Safin via Tarantool-patches
@ 2021-08-06  0:27   ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06  0:27 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

This change dissolved in many patches.

On 02.08.2021 3:41, Timur Safin wrote:
> Renamed all references:
> - `struct datetime_t` -> `struct datetime`;
> - `struct datetime_interval_t` -> `struct datetime_interval`.
> 
> Part of #5941
> ---
>   src/box/tuple_compare.cc    | 10 +++++-----
>   src/lib/core/datetime.c     | 25 ++++++++++++-------------
>   src/lib/core/datetime.h     | 29 ++++++++++++++---------------
>   src/lib/mpstream/mpstream.c |  3 +--
>   src/lib/mpstream/mpstream.h |  5 ++---
>   src/lua/datetime.lua        |  6 +++---
>   src/lua/msgpack.c           |  2 +-
>   src/lua/msgpackffi.lua      |  8 ++++----
>   src/lua/serializer.c        |  2 +-
>   src/lua/serializer.h        |  2 +-
>   src/lua/utils.c             | 10 +++++-----
>   src/lua/utils.h             |  4 ++--
>   test/unit/datetime.c        | 17 +++++++++--------
>   13 files changed, 60 insertions(+), 63 deletions(-)
> 
> diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
> index f733b9f01..530417fcc 100644
> --- a/src/box/tuple_compare.cc
> +++ b/src/box/tuple_compare.cc
> @@ -397,8 +397,8 @@ mp_compare_uuid(const char *field_a, const char *field_b)
>   static int
>   mp_compare_datetime(const char *lhs, const char *rhs)
>   {
> -	datetime_t lhs_dt, rhs_dt;
> -	datetime_t *ret;
> +	datetime lhs_dt, rhs_dt;
> +	datetime *ret;
>   	ret = mp_decode_datetime(&lhs, &lhs_dt);
>   	assert(ret != NULL);
>   	ret = mp_decode_datetime(&rhs, &rhs_dt);
> @@ -1648,7 +1648,7 @@ hint_uuid_raw(const char *data)
>   }
>   
>   static inline hint_t
> -hint_datetime(struct datetime_t *date)
> +hint_datetime(struct datetime *date)
>   {
>   	/*
>   	 * Use at most HINT_VALUE_SECS_BITS from datetime
> @@ -1803,7 +1803,7 @@ field_hint_datetime(const char *field)
>   	int8_t ext_type;
>   	uint32_t len = mp_decode_extl(&field, &ext_type);
>   	assert(ext_type == MP_DATETIME);
> -	struct datetime_t date;
> +	struct datetime date;
>   	return hint_datetime(datetime_unpack(&field, len, &date));
>   }
>   
> @@ -1860,7 +1860,7 @@ field_hint_scalar(const char *field, struct coll *coll)
>   			return hint_uuid_raw(field);
>   		case MP_DATETIME:
>   		{
> -			struct datetime_t date;
> +			struct datetime date;
>   			return hint_datetime(datetime_unpack(&field, len, &date));
>   		}
>   		default:
> diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
> index 96e554bd2..03facb123 100755
> --- a/src/lib/core/datetime.c
> +++ b/src/lib/core/datetime.c
> @@ -84,7 +84,7 @@ mp_decode_Xint(const char **data)
>   }
>   
>   static inline uint32_t
> -mp_sizeof_datetime_raw(const struct datetime_t *date)
> +mp_sizeof_datetime_raw(const struct datetime *date)
>   {
>   	uint32_t sz = mp_sizeof_Xint(date->secs);
>   
> @@ -98,13 +98,13 @@ mp_sizeof_datetime_raw(const struct datetime_t *date)
>   }
>   
>   uint32_t
> -mp_sizeof_datetime(const struct datetime_t *date)
> +mp_sizeof_datetime(const struct datetime *date)
>   {
>   	return mp_sizeof_ext(mp_sizeof_datetime_raw(date));
>   }
>   
> -struct datetime_t *
> -datetime_unpack(const char **data, uint32_t len, struct datetime_t *date)
> +struct datetime *
> +datetime_unpack(const char **data, uint32_t len, struct datetime *date)
>   {
>   	const char * svp = *data;
>   
> @@ -128,8 +128,8 @@ datetime_unpack(const char **data, uint32_t len, struct datetime_t *date)
>   	return date;
>   }
>   
> -struct datetime_t *
> -mp_decode_datetime(const char **data, struct datetime_t *date)
> +struct datetime *
> +mp_decode_datetime(const char **data, struct datetime *date)
>   {
>   	if (mp_typeof(**data) != MP_EXT)
>   		return NULL;
> @@ -144,7 +144,7 @@ mp_decode_datetime(const char **data, struct datetime_t *date)
>   }
>   
>   char *
> -datetime_pack(char *data, const struct datetime_t *date)
> +datetime_pack(char *data, const struct datetime *date)
>   {
>   	data = mp_encode_Xint(data, date->secs);
>   	if (date->nsec != 0 || date->offset != 0)
> @@ -156,7 +156,7 @@ datetime_pack(char *data, const struct datetime_t *date)
>   }
>   
>   char *
> -mp_encode_datetime(char *data, const struct datetime_t *date)
> +mp_encode_datetime(char *data, const struct datetime *date)
>   {
>   	uint32_t len = mp_sizeof_datetime_raw(date);
>   
> @@ -165,8 +165,7 @@ mp_encode_datetime(char *data, const struct datetime_t *date)
>   	return datetime_pack(data, date);
>   }
>   
> -int
> -datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
> +int datetime_to_string(const struct datetime *date, char *buf, uint32_t len)
>   {
>   	char * src = buf;
>   	int offset = date->offset;
> @@ -218,7 +217,7 @@ datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len)
>   int
>   mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
>   {
> -	struct datetime_t date = {0};
> +	struct datetime date = {0};
>   
>   	if (datetime_unpack(data, len, &date) == NULL)
>   		return -1;
> @@ -229,7 +228,7 @@ mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
>   int
>   mp_fprint_datetime(FILE *file, const char **data, uint32_t len)
>   {
> -	struct  datetime_t date;
> +	struct datetime date;
>   
>   	if (datetime_unpack(data, len, &date) == NULL)
>   		return -1;
> @@ -240,7 +239,7 @@ mp_fprint_datetime(FILE *file, const char **data, uint32_t len)
>   	return fprintf(file, "%s", buf);
>   }
>   
> -int datetime_compare(const struct datetime_t *lhs, const struct datetime_t *rhs)
> +int datetime_compare(const struct datetime *lhs, const struct datetime *rhs)
>   {
>   	int result = COMPARE_RESULT(lhs->secs, rhs->secs);
>   	if (result != 0)
> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> index 2c9530ad7..540bd68d9 100644
> --- a/src/lib/core/datetime.h
> +++ b/src/lib/core/datetime.h
> @@ -50,7 +50,8 @@ extern "C"
>    * Time is kept normalized to UTC, time-zone offset
>    * is informative only.
>    */
> -struct datetime_t {
> +struct datetime
> +{
>   	int64_t secs;	/**< seconds since epoch */
>   	int32_t nsec;	/**< nanoseconds if any */
>   	int32_t offset; /**< offset in minutes from UTC */
> @@ -59,42 +60,41 @@ struct datetime_t {
>   /**
>    * Date/time interval structure
>    */
> -struct datetime_interval_t {
> +struct datetime_interval
> +{
>   	int64_t secs; /**< relative seconds delta */
>   	int32_t nsec; /**< nanoseconds delta */
>   };
>   
> -int
> -datetime_compare(const struct datetime_t * lhs,
> -		 const struct datetime_t * rhs);
> +int datetime_compare(const struct datetime *lhs,
> +		     const struct datetime *rhs);
>   
> -
> -struct datetime_t *
> -datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
> +struct datetime *
> +datetime_unpack(const char **data, uint32_t len, struct datetime *date);
>   
>   /**
>    * Pack datetime_t data to the MessagePack buffer.
>    */
>   char *
> -datetime_pack(char *data, const struct datetime_t *date);
> +datetime_pack(char *data, const struct datetime *date);
>   
>   /**
>    * Calculate size of MessagePack buffer for datetime_t data.
>    */
>   uint32_t
> -mp_sizeof_datetime(const struct datetime_t *date);
> +mp_sizeof_datetime(const struct datetime *date);
>   
>   /**
>    * Decode data from MessagePack buffer to datetime_t structure.
>    */
> -struct datetime_t *
> -mp_decode_datetime(const char **data, struct datetime_t *date);
> +struct datetime *
> +mp_decode_datetime(const char **data, struct datetime *date);
>   
>   /**
>    * Encode datetime_t structure to the MessagePack buffer.
>    */
>   char *
> -mp_encode_datetime(char *data, const struct datetime_t *date);
> +mp_encode_datetime(char *data, const struct datetime *date);
>   
>   /**
>    * Convert datetime to string using default format
> @@ -102,8 +102,7 @@ mp_encode_datetime(char *data, const struct datetime_t *date);
>    * @param buf output character buffer
>    * @param len size ofoutput buffer
>    */
> -int
> -datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
> +int datetime_to_string(const struct datetime *date, char *buf, uint32_t len);
>   
>   int
>   mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
> diff --git a/src/lib/mpstream/mpstream.c b/src/lib/mpstream/mpstream.c
> index 1077e3b19..2af77c205 100644
> --- a/src/lib/mpstream/mpstream.c
> +++ b/src/lib/mpstream/mpstream.c
> @@ -209,8 +209,7 @@ mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid)
>   	mpstream_advance(stream, pos - data);
>   }
>   
> -void
> -mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *val)
> +void mpstream_encode_datetime(struct mpstream *stream, const struct datetime *val)
>   {
>   	char *data = mpstream_reserve(stream, mp_sizeof_datetime(val));
>   	if (data == NULL)
> diff --git a/src/lib/mpstream/mpstream.h b/src/lib/mpstream/mpstream.h
> index 540e9a666..9e0ed6e72 100644
> --- a/src/lib/mpstream/mpstream.h
> +++ b/src/lib/mpstream/mpstream.h
> @@ -39,7 +39,7 @@ extern "C" {
>   #endif /* defined(__cplusplus) */
>   
>   struct tt_uuid;
> -struct datetime_t;
> +struct datetime;
>   
>   /**
>   * Ask the allocator to reserve at least size bytes. It can reserve
> @@ -146,8 +146,7 @@ mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val);
>   void
>   mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid);
>   
> -void
> -mpstream_encode_datetime(struct mpstream *stream, const struct datetime_t *dt);
> +void mpstream_encode_datetime(struct mpstream *stream, const struct datetime *dt);
>   
>   /** Copies n bytes from memory area src to stream. */
>   void
> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> index dc88a9d9d..5cad4e02f 100644
> --- a/src/lua/datetime.lua
> +++ b/src/lua/datetime.lua
> @@ -80,7 +80,7 @@ ffi.cdef [[
>       // datetime.c
>   
>       int
> -    datetime_to_string(const struct datetime_t * date, char *buf, uint32_t len);
> +    datetime_to_string(const struct datetime * date, char *buf, uint32_t len);
>   
>   
>       // <asm-generic/posix_types.h>
> @@ -180,8 +180,8 @@ local NANOS_PER_SEC    = 1000000000LL
>   local DT_EPOCH_1970_OFFSET = 719163LL
>   
>   
> -local datetime_t = ffi.typeof('struct datetime_t')
> -local interval_t = ffi.typeof('struct datetime_interval_t')
> +local datetime_t = ffi.typeof('struct datetime')
> +local interval_t = ffi.typeof('struct datetime_interval')
>   ffi.cdef [[
>       struct t_interval_months {
>           int m;
> diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
> index 0a4ba8129..9c1acc4dc 100644
> --- a/src/lua/msgpack.c
> +++ b/src/lua/msgpack.c
> @@ -339,7 +339,7 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
>   		}
>   		case MP_DATETIME:
>   		{
> -			struct datetime_t * date = luaL_pushdatetime(L);
> +			struct datetime *date = luaL_pushdatetime(L);
>   			date = datetime_unpack(data, len, date);
>   			if (date == NULL)
>   				goto ext_decode_err;
> diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua
> index c47d77acb..fb5e7d644 100644
> --- a/src/lua/msgpackffi.lua
> +++ b/src/lua/msgpackffi.lua
> @@ -40,8 +40,8 @@ decimal_t *
>   decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
>   struct tt_uuid *
>   uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid);
> -struct datetime_t *
> -datetime_unpack(const char **data, uint32_t len, struct datetime_t *date);
> +struct datetime *
> +datetime_unpack(const char **data, uint32_t len, struct datetime *date);
>   ]])
>   
>   local strict_alignment = (jit.arch == 'arm')
> @@ -331,7 +331,7 @@ on_encode(ffi.typeof('float'), encode_float)
>   on_encode(ffi.typeof('double'), encode_double)
>   on_encode(ffi.typeof('decimal_t'), encode_decimal)
>   on_encode(ffi.typeof('struct tt_uuid'), encode_uuid)
> -on_encode(ffi.typeof('struct datetime_t'), encode_datetime)
> +on_encode(ffi.typeof('struct datetime'), encode_datetime)
>   
>   --------------------------------------------------------------------------------
>   -- Decoder
> @@ -527,7 +527,7 @@ local ext_decoder = {
>       end,
>       -- MP_DATETIME
>       [4] = function(data, len)
> -        local dt = ffi.new("struct datetime_t")
> +        local dt = ffi.new("struct datetime")
>           builtin.datetime_unpack(data, len, dt)
>           return dt
>       end,
> diff --git a/src/lua/serializer.c b/src/lua/serializer.c
> index c27e62c62..24f4a5ff9 100644
> --- a/src/lua/serializer.c
> +++ b/src/lua/serializer.c
> @@ -547,7 +547,7 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg,
>   				field->ext_type = MP_ERROR;
>   			} else if (cd->ctypeid == CTID_DATETIME) {
>   				field->ext_type = MP_DATETIME;
> -				field->dateval = (struct datetime_t*) cdata;
> +				field->dateval = (struct datetime *)cdata;
>   			} else {
>   				field->ext_type = MP_UNKNOWN_EXTENSION;
>   			}
> diff --git a/src/lua/serializer.h b/src/lua/serializer.h
> index 52e51d279..e7a240e0a 100644
> --- a/src/lua/serializer.h
> +++ b/src/lua/serializer.h
> @@ -224,7 +224,7 @@ struct luaL_field {
>   		uint32_t size;
>   		decimal_t *decval;
>   		struct tt_uuid *uuidval;
> -		struct datetime_t *dateval;
> +		struct datetime *dateval;
>   	};
>   	enum mp_type type;
>   	/* subtypes of MP_EXT */
> diff --git a/src/lua/utils.c b/src/lua/utils.c
> index 685cab47b..9753016c9 100644
> --- a/src/lua/utils.c
> +++ b/src/lua/utils.c
> @@ -110,7 +110,7 @@ luaL_pushuuid(struct lua_State *L)
>   	return luaL_pushcdata(L, CTID_UUID);
>   }
>   
> -struct datetime_t *
> +struct datetime *
>   luaL_pushdatetime(struct lua_State *L)
>   {
>   	return luaL_pushcdata(L, CTID_DATETIME);
> @@ -720,22 +720,22 @@ tarantool_lua_utils_init(struct lua_State *L)
>   	CTID_UUID = luaL_ctypeid(L, "struct tt_uuid");
>   	assert(CTID_UUID != 0);
>   
> -	rc = luaL_cdef(L, "struct datetime_t {"
> +	rc = luaL_cdef(L, "struct datetime {"
>   			  "int64_t secs;"
>   			  "int32_t nsec;"
>   			  "int32_t offset;"
>   			  "};");
>   	assert(rc == 0);
>   	(void) rc;
> -	CTID_DATETIME = luaL_ctypeid(L, "struct datetime_t");
> +	CTID_DATETIME = luaL_ctypeid(L, "struct datetime");
>   	assert(CTID_DATETIME != 0);
> -	rc = luaL_cdef(L, "struct datetime_interval_t {"
> +	rc = luaL_cdef(L, "struct datetime_interval {"
>   			  "int64_t secs;"
>   			  "int32_t nsec;"
>   			  "};");
>   	assert(rc == 0);
>   	(void) rc;
> -	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval_t");
> +	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval");
>   	assert(CTID_INTERVAL != 0);
>   
>   	lua_pushcfunction(L, luaT_newthread_wrapper);
> diff --git a/src/lua/utils.h b/src/lua/utils.h
> index bf56b1bb5..c27065a1c 100644
> --- a/src/lua/utils.h
> +++ b/src/lua/utils.h
> @@ -59,7 +59,7 @@ struct lua_State;
>   struct ibuf;
>   typedef struct ibuf box_ibuf_t;
>   struct tt_uuid;
> -struct datetime_t;
> +struct datetime;
>   
>   /**
>    * Single global lua_State shared by core and modules.
> @@ -84,7 +84,7 @@ luaL_pushuuid(struct lua_State *L);
>    * @sa luaL_pushcdata
>    * @return memory associated with this datetime_t data
>    */
> -struct datetime_t *
> +struct datetime *
>   luaL_pushdatetime(struct lua_State *L);
>   
>   /** \cond public */
> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
> index 226b6fadb..e8a022542 100644
> --- a/test/unit/datetime.c
> +++ b/test/unit/datetime.c
> @@ -141,26 +141,27 @@ exit:
>   #define SECS_PER_DAY      86400
>   #define DT_EPOCH_1970_OFFSET 719163
>   
> -
> -struct datetime_t {
> +struct datetime
> +{
>   	int64_t secs;
>   	int32_t nsec;
>   	int32_t offset;
>   };
>   
>   static int
> -local_rd(const struct datetime_t * dt) {
> +local_rd(const struct datetime *dt)
> +{
>   	return (int)(dt->secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET;
>   }
>   
>   static int
> -local_dt(const struct datetime_t * dt) {
> +local_dt(const struct datetime *dt)
> +{
>   	return dt_from_rdn(local_rd(dt));
>   }
>   
> -
> -struct tm*
> -datetime_to_tm(struct datetime_t * dt)
> +struct tm *
> +datetime_to_tm(struct datetime *dt)
>   {
>   	static struct tm tm;
>   
> @@ -198,7 +199,7 @@ static void datetime_test(void)
>   		// check that stringized literal produces the same date
>   		// time fields
>   		static char buff[40];
> -		struct datetime_t dt = {secs, nanosecs, ofs};
> +		struct datetime dt = {secs, nanosecs, ofs};
>   		// datetime_to_tm returns time in GMT zone
>   		struct tm * p_tm = datetime_to_tm(&dt);
>   		size_t len = strftime(buff, sizeof buff, "%F %T%z", p_tm);
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 1/2] datetime: update tests for macosx
  2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 1/2] datetime: update tests for macosx Timur Safin via Tarantool-patches
@ 2021-08-06  0:28   ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06  0:28 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

This guy got integrated to the 1/9 patch also, where we now introduce 
unit-test.


On 04.08.2021 0:23, Timur Safin wrote:
> `strftime()` `%z` outputs local time-zone if `tm_gmtoff` is 0.
> This behaviour is different to that we observe on Linux, thus we
> have different execution results. Updated test to not use `%z`, and only
> operate with normalized date time formats `%F` and `%T`
> 
> Part of #5941
> ---
>   test/unit/datetime.c      |   2 +-
>   test/unit/datetime.result | 284 +++++++++++++++++++-------------------
>   2 files changed, 143 insertions(+), 143 deletions(-)
> 
> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
> index e8a022542..95df7ce68 100644
> --- a/test/unit/datetime.c
> +++ b/test/unit/datetime.c
> @@ -202,7 +202,7 @@ static void datetime_test(void)
>   		struct datetime dt = {secs, nanosecs, ofs};
>   		// datetime_to_tm returns time in GMT zone
>   		struct tm * p_tm = datetime_to_tm(&dt);
> -		size_t len = strftime(buff, sizeof buff, "%F %T%z", p_tm);
> +		size_t len = strftime(buff, sizeof buff, "%F %T", p_tm);
>   		ok(len > 0, "strftime");
>   		rc = parse_datetime(buff, len, &dt.secs, &dt.nsec, &dt.offset);
>   		is(rc, 0, "correct parse_datetime return value for '%s'", buff);
> diff --git a/test/unit/datetime.result b/test/unit/datetime.result
> index f106fa769..33997d9df 100644
> --- a/test/unit/datetime.result
> +++ b/test/unit/datetime.result
> @@ -3,356 +3,356 @@
>       ok 1 - correct parse_datetime return value for '2012-12-24 15:30Z'
>       ok 2 - correct parse_datetime output seconds for '2012-12-24 15:30Z
>       ok 3 - strftime
> -    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 6 - correct parse_datetime return value for '2012-12-24 15:30z'
>       ok 7 - correct parse_datetime output seconds for '2012-12-24 15:30z
>       ok 8 - strftime
> -    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 11 - correct parse_datetime return value for '2012-12-24 15:30'
>       ok 12 - correct parse_datetime output seconds for '2012-12-24 15:30
>       ok 13 - strftime
> -    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 16 - correct parse_datetime return value for '2012-12-24 16:30+01:00'
>       ok 17 - correct parse_datetime output seconds for '2012-12-24 16:30+01:00
>       ok 18 - strftime
> -    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 21 - correct parse_datetime return value for '2012-12-24 16:30+0100'
>       ok 22 - correct parse_datetime output seconds for '2012-12-24 16:30+0100
>       ok 23 - strftime
> -    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 26 - correct parse_datetime return value for '2012-12-24 16:30+01'
>       ok 27 - correct parse_datetime output seconds for '2012-12-24 16:30+01
>       ok 28 - strftime
> -    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 31 - correct parse_datetime return value for '2012-12-24 14:30-01:00'
>       ok 32 - correct parse_datetime output seconds for '2012-12-24 14:30-01:00
>       ok 33 - strftime
> -    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 36 - correct parse_datetime return value for '2012-12-24 14:30-0100'
>       ok 37 - correct parse_datetime output seconds for '2012-12-24 14:30-0100
>       ok 38 - strftime
> -    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 41 - correct parse_datetime return value for '2012-12-24 14:30-01'
>       ok 42 - correct parse_datetime output seconds for '2012-12-24 14:30-01
>       ok 43 - strftime
> -    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 46 - correct parse_datetime return value for '2012-12-24 15:30:00Z'
>       ok 47 - correct parse_datetime output seconds for '2012-12-24 15:30:00Z
>       ok 48 - strftime
> -    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 51 - correct parse_datetime return value for '2012-12-24 15:30:00z'
>       ok 52 - correct parse_datetime output seconds for '2012-12-24 15:30:00z
>       ok 53 - strftime
> -    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 56 - correct parse_datetime return value for '2012-12-24 15:30:00'
>       ok 57 - correct parse_datetime output seconds for '2012-12-24 15:30:00
>       ok 58 - strftime
> -    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 61 - correct parse_datetime return value for '2012-12-24 16:30:00+01:00'
>       ok 62 - correct parse_datetime output seconds for '2012-12-24 16:30:00+01:00
>       ok 63 - strftime
> -    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 66 - correct parse_datetime return value for '2012-12-24 16:30:00+0100'
>       ok 67 - correct parse_datetime output seconds for '2012-12-24 16:30:00+0100
>       ok 68 - strftime
> -    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 71 - correct parse_datetime return value for '2012-12-24 14:30:00-01:00'
>       ok 72 - correct parse_datetime output seconds for '2012-12-24 14:30:00-01:00
>       ok 73 - strftime
> -    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 76 - correct parse_datetime return value for '2012-12-24 14:30:00-0100'
>       ok 77 - correct parse_datetime output seconds for '2012-12-24 14:30:00-0100
>       ok 78 - strftime
> -    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 81 - correct parse_datetime return value for '2012-12-24 15:30:00.123456Z'
>       ok 82 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456Z
>       ok 83 - strftime
> -    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 86 - correct parse_datetime return value for '2012-12-24 15:30:00.123456z'
>       ok 87 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456z
>       ok 88 - strftime
> -    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 91 - correct parse_datetime return value for '2012-12-24 15:30:00.123456'
>       ok 92 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456
>       ok 93 - strftime
> -    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 96 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01:00'
>       ok 97 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01:00
>       ok 98 - strftime
> -    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 101 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01'
>       ok 102 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01
>       ok 103 - strftime
> -    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 106 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01:00'
>       ok 107 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01:00
>       ok 108 - strftime
> -    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 111 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01'
>       ok 112 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01
>       ok 113 - strftime
> -    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 116 - correct parse_datetime return value for '2012-12-24t15:30Z'
>       ok 117 - correct parse_datetime output seconds for '2012-12-24t15:30Z
>       ok 118 - strftime
> -    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 121 - correct parse_datetime return value for '2012-12-24t15:30z'
>       ok 122 - correct parse_datetime output seconds for '2012-12-24t15:30z
>       ok 123 - strftime
> -    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 126 - correct parse_datetime return value for '2012-12-24t15:30'
>       ok 127 - correct parse_datetime output seconds for '2012-12-24t15:30
>       ok 128 - strftime
> -    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 131 - correct parse_datetime return value for '2012-12-24t16:30+01:00'
>       ok 132 - correct parse_datetime output seconds for '2012-12-24t16:30+01:00
>       ok 133 - strftime
> -    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 136 - correct parse_datetime return value for '2012-12-24t16:30+0100'
>       ok 137 - correct parse_datetime output seconds for '2012-12-24t16:30+0100
>       ok 138 - strftime
> -    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 141 - correct parse_datetime return value for '2012-12-24t14:30-01:00'
>       ok 142 - correct parse_datetime output seconds for '2012-12-24t14:30-01:00
>       ok 143 - strftime
> -    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 146 - correct parse_datetime return value for '2012-12-24t14:30-0100'
>       ok 147 - correct parse_datetime output seconds for '2012-12-24t14:30-0100
>       ok 148 - strftime
> -    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 151 - correct parse_datetime return value for '2012-12-24t15:30:00Z'
>       ok 152 - correct parse_datetime output seconds for '2012-12-24t15:30:00Z
>       ok 153 - strftime
> -    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 156 - correct parse_datetime return value for '2012-12-24t15:30:00z'
>       ok 157 - correct parse_datetime output seconds for '2012-12-24t15:30:00z
>       ok 158 - strftime
> -    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 161 - correct parse_datetime return value for '2012-12-24t15:30:00'
>       ok 162 - correct parse_datetime output seconds for '2012-12-24t15:30:00
>       ok 163 - strftime
> -    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 166 - correct parse_datetime return value for '2012-12-24t16:30:00+01:00'
>       ok 167 - correct parse_datetime output seconds for '2012-12-24t16:30:00+01:00
>       ok 168 - strftime
> -    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 171 - correct parse_datetime return value for '2012-12-24t16:30:00+0100'
>       ok 172 - correct parse_datetime output seconds for '2012-12-24t16:30:00+0100
>       ok 173 - strftime
> -    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 176 - correct parse_datetime return value for '2012-12-24t14:30:00-01:00'
>       ok 177 - correct parse_datetime output seconds for '2012-12-24t14:30:00-01:00
>       ok 178 - strftime
> -    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 181 - correct parse_datetime return value for '2012-12-24t14:30:00-0100'
>       ok 182 - correct parse_datetime output seconds for '2012-12-24t14:30:00-0100
>       ok 183 - strftime
> -    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 186 - correct parse_datetime return value for '2012-12-24t15:30:00.123456Z'
>       ok 187 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456Z
>       ok 188 - strftime
> -    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 191 - correct parse_datetime return value for '2012-12-24t15:30:00.123456z'
>       ok 192 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456z
>       ok 193 - strftime
> -    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 196 - correct parse_datetime return value for '2012-12-24t16:30:00.123456+01:00'
>       ok 197 - correct parse_datetime output seconds for '2012-12-24t16:30:00.123456+01:00
>       ok 198 - strftime
> -    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 201 - correct parse_datetime return value for '2012-12-24t14:30:00.123456-01:00'
>       ok 202 - correct parse_datetime output seconds for '2012-12-24t14:30:00.123456-01:00
>       ok 203 - strftime
> -    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 206 - correct parse_datetime return value for '2012-12-24 16:30 +01:00'
>       ok 207 - correct parse_datetime output seconds for '2012-12-24 16:30 +01:00
>       ok 208 - strftime
> -    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 211 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
>       ok 212 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
>       ok 213 - strftime
> -    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 216 - correct parse_datetime return value for '2012-12-24 15:30 UTC'
>       ok 217 - correct parse_datetime output seconds for '2012-12-24 15:30 UTC
>       ok 218 - strftime
> -    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 221 - correct parse_datetime return value for '2012-12-24 16:30 UTC+1'
>       ok 222 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+1
>       ok 223 - strftime
> -    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 226 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01'
>       ok 227 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01
>       ok 228 - strftime
> -    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 231 - correct parse_datetime return value for '2012-12-24 16:30 UTC+0100'
>       ok 232 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+0100
>       ok 233 - strftime
> -    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 236 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01:00'
>       ok 237 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01:00
>       ok 238 - strftime
> -    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 241 - correct parse_datetime return value for '2012-12-24 14:30 UTC-1'
>       ok 242 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-1
>       ok 243 - strftime
> -    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 246 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01'
>       ok 247 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01
>       ok 248 - strftime
> -    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 251 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01:00'
>       ok 252 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01:00
>       ok 253 - strftime
> -    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 256 - correct parse_datetime return value for '2012-12-24 14:30 UTC-0100'
>       ok 257 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-0100
>       ok 258 - strftime
> -    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 261 - correct parse_datetime return value for '2012-12-24 15:30 GMT'
>       ok 262 - correct parse_datetime output seconds for '2012-12-24 15:30 GMT
>       ok 263 - strftime
> -    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 266 - correct parse_datetime return value for '2012-12-24 16:30 GMT+1'
>       ok 267 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+1
>       ok 268 - strftime
> -    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 271 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01'
>       ok 272 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01
>       ok 273 - strftime
> -    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 276 - correct parse_datetime return value for '2012-12-24 16:30 GMT+0100'
>       ok 277 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+0100
>       ok 278 - strftime
> -    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 281 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01:00'
>       ok 282 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01:00
>       ok 283 - strftime
> -    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 286 - correct parse_datetime return value for '2012-12-24 14:30 GMT-1'
>       ok 287 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-1
>       ok 288 - strftime
> -    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 291 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01'
>       ok 292 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01
>       ok 293 - strftime
> -    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 296 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01:00'
>       ok 297 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01:00
>       ok 298 - strftime
> -    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 301 - correct parse_datetime return value for '2012-12-24 14:30 GMT-0100'
>       ok 302 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-0100
>       ok 303 - strftime
> -    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 306 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
>       ok 307 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
>       ok 308 - strftime
> -    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 311 - correct parse_datetime return value for '2012-12-24 16:30:00 +01:00'
>       ok 312 - correct parse_datetime output seconds for '2012-12-24 16:30:00 +01:00
>       ok 313 - strftime
> -    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 316 - correct parse_datetime return value for '2012-12-24 14:30:00 -01:00'
>       ok 317 - correct parse_datetime output seconds for '2012-12-24 14:30:00 -01:00
>       ok 318 - strftime
> -    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 321 - correct parse_datetime return value for '2012-12-24 16:30:00.123456 +01:00'
>       ok 322 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456 +01:00
>       ok 323 - strftime
> -    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 326 - correct parse_datetime return value for '2012-12-24 14:30:00.123456 -01:00'
>       ok 327 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456 -01:00
>       ok 328 - strftime
> -    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 331 - correct parse_datetime return value for '2012-12-24 15:30:00.123456 -00:00'
>       ok 332 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456 -00:00
>       ok 333 - strftime
> -    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 336 - correct parse_datetime return value for '20121224T1630+01:00'
>       ok 337 - correct parse_datetime output seconds for '20121224T1630+01:00
>       ok 338 - strftime
> -    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 341 - correct parse_datetime return value for '2012-12-24T1630+01:00'
>       ok 342 - correct parse_datetime output seconds for '2012-12-24T1630+01:00
>       ok 343 - strftime
> -    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 346 - correct parse_datetime return value for '20121224T16:30+01'
>       ok 347 - correct parse_datetime output seconds for '20121224T16:30+01
>       ok 348 - strftime
> -    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00
>       ok 351 - correct parse_datetime return value for '20121224T16:30 +01'
>       ok 352 - correct parse_datetime output seconds for '20121224T16:30 +01
>       ok 353 - strftime
> -    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00+0000'
> -    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00+0000
> +    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00'
> +    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00
>   ok 1 - subtests
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers
  2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers Timur Safin via Tarantool-patches
@ 2021-08-06  0:30   ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06  0:30 UTC (permalink / raw)
  To: v.shpilevoy, olegrok; +Cc: tarantool-patches

This change is now part of patch 2/9 lua:built-in module datetime, and 
there is no platform specific Lua code in the module.

Timur

On 04.08.2021 0:23, Timur Safin wrote:
> To make Lua code less platfrom dependent we move
> all platform specifics to the C level. Specifically
> we want to avoid dependence on `struct tm {}` layout
> and GLIBC `strftime` behaviour.
> As a bonus we have that after moving of time dependent
> code to C we make Lua code significantly simpler and thus
> may get rid of all implementation specific details
> with numerous typedefs and function declarations.
> 
> Part of #5941
> ---
>   src/exports.h           |   4 ++
>   src/lib/core/datetime.c |  63 +++++++++++++++++-
>   src/lib/core/datetime.h |  23 ++++++-
>   src/lua/datetime.lua    | 139 +++++-----------------------------------
>   4 files changed, 104 insertions(+), 125 deletions(-)
> 
> diff --git a/src/exports.h b/src/exports.h
> index 63efe0ec7..f4a6f98d7 100644
> --- a/src/exports.h
> +++ b/src/exports.h
> @@ -218,6 +218,10 @@ EXPORT(curl_version)
>   EXPORT(curl_version_info)
>   #endif /* EXPORT_LIBCURL_SYMBOLS */
>   EXPORT(datetime_pack)
> +EXPORT(datetime_asctime)
> +EXPORT(datetime_ctime)
> +EXPORT(datetime_now)
> +EXPORT(datetime_strftime)
>   EXPORT(datetime_to_string)
>   EXPORT(datetime_unpack)
>   EXPORT(decimal_unpack)
> diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
> index 03facb123..a921cbd11 100755
> --- a/src/lib/core/datetime.c
> +++ b/src/lib/core/datetime.c
> @@ -30,6 +30,7 @@
>    */
>   
>   #include <string.h>
> +#include <time.h>
>   
>   #include "trivia/util.h"
>   #include "datetime.h"
> @@ -165,12 +166,35 @@ mp_encode_datetime(char *data, const struct datetime *date)
>   	return datetime_pack(data, date);
>   }
>   
> +static int
> +local_dt(int64_t secs)
> +{
> +	return dt_from_rdn((int)(secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET);
> +}
> +
> +static struct tm *
> +datetime_to_tm(const struct datetime *date)
> +{
> +	static struct tm tm;
> +
> +	memset(&tm, 0, sizeof(tm));
> +	int64_t secs = date->secs;
> +	dt_to_struct_tm(local_dt(secs), &tm);
> +
> +	int seconds_of_day = date->secs % SECS_PER_DAY;
> +	tm.tm_hour = (seconds_of_day / 3600) % 24;
> +	tm.tm_min = (seconds_of_day / 60) % 60;
> +	tm.tm_sec = seconds_of_day % 60;
> +
> +	return &tm;
> +}
> +
>   int datetime_to_string(const struct datetime *date, char *buf, uint32_t len)
>   {
>   	char * src = buf;
>   	int offset = date->offset;
>   	int64_t secs = date->secs + offset * 60;
> -	dt_t dt = dt_from_rdn((secs / SECS_PER_DAY) + 719163);
> +	dt_t dt = local_dt(secs);
>   
>   	int year, month, day, sec, ns, sign;
>   	dt_to_ymd(dt, &year, &month, &day);
> @@ -214,6 +238,43 @@ int datetime_to_string(const struct datetime *date, char *buf, uint32_t len)
>   	return (buf - src);
>   }
>   
> +void
> +datetime_now(struct datetime * now)
> +{
> +	struct timeval tv;
> +	gettimeofday(&tv, NULL);
> +	now->secs = tv.tv_sec;
> +	now->nsec = tv.tv_usec * 1000;
> +
> +	time_t now_seconds;
> +	time(&now_seconds);
> +	struct tm tm;
> +	localtime_r(&now_seconds, &tm);
> +	now->offset = tm.tm_gmtoff / 60;
> +}
> +
> +char *
> +datetime_asctime(const struct datetime *date)
> +{
> +	struct tm *p_tm = datetime_to_tm(date);
> +	return asctime(p_tm);
> +}
> +
> +char *
> +datetime_ctime(const struct datetime *date)
> +{
> +	time_t time = date->secs;
> +	return ctime(&time);
> +}
> +
> +size_t
> +datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +		  uint32_t len)
> +{
> +	struct tm *p_tm = datetime_to_tm(date);
> +	return strftime(buf, len, fmt, p_tm);
> +}
> +
>   int
>   mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
>   {
> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> index 540bd68d9..012fb340d 100644
> --- a/src/lib/core/datetime.h
> +++ b/src/lib/core/datetime.h
> @@ -42,6 +42,7 @@ extern "C"
>   
>   #ifndef SECS_PER_DAY
>   #define SECS_PER_DAY	86400
> +#define DT_EPOCH_1970_OFFSET 719163
>   #endif
>   
>   /**
> @@ -102,7 +103,27 @@ mp_encode_datetime(char *data, const struct datetime *date);
>    * @param buf output character buffer
>    * @param len size ofoutput buffer
>    */
> -int datetime_to_string(const struct datetime *date, char *buf, uint32_t len);
> +int
> +datetime_to_string(const struct datetime *date, char *buf, uint32_t len);
> +
> +/**
> + * Convert datetime to string using default asctime format
> + * "Sun Sep 16 01:03:52 1973\n\0"
> + * @param date source datetime value
> + * @sa datetime_ctime
> + */
> +char *
> +datetime_asctime(const struct datetime *date);
> +
> +char *
> +datetime_ctime(const struct datetime *date);
> +
> +size_t
> +datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +		  uint32_t len);
> +
> +void
> +datetime_now(struct datetime * now);
>   
>   int
>   mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> index 5cad4e02f..1549735bd 100644
> --- a/src/lua/datetime.lua
> +++ b/src/lua/datetime.lua
> @@ -82,90 +82,18 @@ ffi.cdef [[
>       int
>       datetime_to_string(const struct datetime * date, char *buf, uint32_t len);
>   
> +    char *
> +    datetime_asctime(const struct datetime *date);
>   
> -    // <asm-generic/posix_types.h>
> -    typedef long            __kernel_long_t;
> -    typedef unsigned long   __kernel_ulong_t;
> -    // /usr/include/x86_64-linux-gnu/bits/types/time_t.h
> -    typedef long            time_t;
> +    char *
> +    datetime_ctime(const struct datetime *date);
>   
> +    size_t
> +    datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +                      uint32_t len);
>   
> -    // <time.h>
> -    typedef __kernel_long_t	__kernel_time_t;
> -    typedef __kernel_long_t	__kernel_suseconds_t;
> -
> -    struct timespec {
> -        __kernel_time_t	        tv_sec;     /* seconds */
> -        long                    tv_nsec;    /* nanoseconds */
> -    };
> -
> -    struct timeval {
> -        __kernel_time_t	        tv_sec;	    /* seconds */
> -        __kernel_suseconds_t    tv_usec;    /* microseconds */
> -    };
> -
> -    struct timezone {
> -        int	tz_minuteswest;     /* minutes west of Greenwich */
> -        int	tz_dsttime;	        /* type of dst correction */
> -    };
> -
> -    // /usr/include/x86_64-linux-gnu/sys/time.h
> -    typedef struct timezone * __timezone_ptr_t;
> -
> -    /* Get the current time of day and timezone information,
> -       putting it into *TV and *TZ.  If TZ is NULL, *TZ is not filled.
> -       Returns 0 on success, -1 on errors.
> -
> -       NOTE: This form of timezone information is obsolete.
> -       Use the functions and variables declared in <time.h> instead.  */
> -    int gettimeofday (struct timeval *__tv, struct timezone * __tz);
> -
> -    // /usr/include/x86_64-linux-gnu/bits/types/struct_tm.h
> -    /* ISO C `broken-down time' structure.  */
> -    struct tm
> -    {
> -        int tm_sec;	        /* Seconds.	[0-60] (1 leap second) */
> -        int tm_min;	        /* Minutes.	[0-59] */
> -        int tm_hour;        /* Hours.	[0-23] */
> -        int tm_mday;        /* Day.		[1-31] */
> -        int tm_mon;	        /* Month.	[0-11] */
> -        int tm_year;        /* Year	- 1900.  */
> -        int tm_wday;        /* Day of week.	[0-6] */
> -        int tm_yday;        /* Days in year.[0-365]	*/
> -        int tm_isdst;       /* DST.		[-1/0/1]*/
> -
> -        long int tm_gmtoff; /* Seconds east of UTC.  */
> -        const char *tm_zone;/* Timezone abbreviation.  */
> -    };
> -
> -    // <time.h>
> -    /* Return the current time and put it in *TIMER if TIMER is not NULL.  */
> -    time_t time (time_t *__timer);
> -
> -    /* Format TP into S according to FORMAT.
> -    Write no more than MAXSIZE characters and return the number
> -    of characters written, or 0 if it would exceed MAXSIZE.  */
> -    size_t strftime (char * __s, size_t __maxsize, const char * __format,
> -                     const struct tm * __tp);
> -
> -    /* Parse S according to FORMAT and store binary time information in TP.
> -    The return value is a pointer to the first unparsed character in S.  */
> -    char *strptime (const char * __s, const char * __fmt, struct tm *__tp);
> -
> -    /* Return the `struct tm' representation of *TIMER in UTC,
> -    using *TP to store the result.  */
> -    struct tm *gmtime_r (const time_t * __timer, struct tm * __tp);
> -
> -    /* Return the `struct tm' representation of *TIMER in local time,
> -    using *TP to store the result.  */
> -    struct tm *localtime_r (const time_t * __timer, struct tm * __tp);
> -
> -    /* Return a string of the form "Day Mon dd hh:mm:ss yyyy\n"
> -    that is the representation of TP in this format.  */
> -    char *asctime (const struct tm *__tp);
> -
> -    /* Equivalent to `asctime (localtime (timer))'.  */
> -    char *ctime (const time_t *__timer);
> +    void
> +    datetime_now(struct datetime * now);
>   
>   ]]
>   
> @@ -921,62 +849,27 @@ local function datetime_from(o)
>   end
>   
>   local function local_now()
> -    local p_tv = ffi.new('struct timeval [1]')
> -    local rc = builtin.gettimeofday(p_tv, nil)
> -    assert(rc == 0)
> -
> -    local secs = p_tv[0].tv_sec
> -    local nsec = p_tv[0].tv_usec * 1000
> -
> -    local p_time = ffi.new('time_t[1]')
> -    local p_tm = ffi.new('struct tm[1]')
> -    builtin.time(p_time)
> -    builtin.localtime_r(p_time, p_tm)
> -    local ofs = p_tm[0].tm_gmtoff / 60 -- convert seconds to minutes
> -
> -    return datetime_new_raw(secs, nsec, ofs)
> -end
> -
> -local function datetime_to_tm_ptr(o)
> -    assert(is_datetime(o))
> -    local p_tm = ffi.new('struct tm[1]')
> -    -- dt_to_struct_tm() fills only date data
> -    builtin.dt_to_struct_tm(local_dt(o), p_tm)
> -
> -    -- calculate the smaller data (hour, minute,
> -    -- seconds) using datetime seconds value
> -    local seconds_of_day = o.secs % 86400
> -    local hour = (seconds_of_day / 3600) % 24
> -    local minute = (seconds_of_day / 60) % 60
> -    p_tm[0].tm_sec = seconds_of_day % 60
> -    p_tm[0].tm_min = minute
> -    p_tm[0].tm_hour = hour
> -
> -    p_tm[0].tm_gmtoff = o.offset * 60
> -
> -    return p_tm
> +    local d = datetime_new_raw(0, 0, 0)
> +    builtin.datetime_now(d)
> +    return d
>   end
>   
>   local function asctime(o)
>       check_date(o, "datetime:asctime()")
>   
> -    local p_tm = datetime_to_tm_ptr(o)
> -    return ffi.string(builtin.asctime(p_tm))
> +    return ffi.string(builtin.datetime_asctime(o))
>   end
>   
>   local function ctime(o)
>       check_date(o, "datetime:ctime()")
> -    local p_time = ffi.new('time_t[1]')
> -    p_time[0] = o.secs
> -    return ffi.string(builtin.ctime(p_time))
> +    return ffi.string(builtin.datetime_ctime(o))
>   end
>   
>   local function strftime(fmt, o)
>       check_date(o, "datetime.strftime()")
> -    local p_tm = datetime_to_tm_ptr(o)
> -    local sz = builtin.strftime(nil, 1024, fmt, p_tm) + 1
> +    local sz = 128
>       local buff = ffi.new('char[?]', sz)
> -    builtin.strftime(buff, sz, fmt, p_tm)
> +    builtin.datetime_strftime(o, fmt, buff, sz)
>       return ffi.string(buff)
>   end
>   
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-06  0:23     ` Safin Timur via Tarantool-patches
@ 2021-08-06  1:30       ` Oleg Babin via Tarantool-patches
  2021-08-06 13:00         ` Safin Timur via Tarantool-patches
  2021-08-08 11:26       ` Vladislav Shpilevoy via Tarantool-patches
  1 sibling, 1 reply; 38+ messages in thread
From: Oleg Babin via Tarantool-patches @ 2021-08-06  1:30 UTC (permalink / raw)
  To: Safin Timur, Vladislav Shpilevoy; +Cc: tarantool-patches


On 06.08.2021 03:23, Safin Timur wrote:
>
> i.e. whenever we see not datetime or interva value datetime_cmp return 
> nil, and all comparisons always return false. Please give me know if 
> that's incorrect scenario and we should raise exception. This was the 
> way Oleg Bain has requested it to behave. (return false for bogus data)
>
Hi! Seems you understood me wrong. It was only about equal comparison.

```

tarantool> uuid.new() == newproxy()
---
- false
...

tarantool> uuid.new() > newproxy()
---
- error: '[string "return require(''uuid'').new() > newproxy()"]:1: 
incorrect value
     to convert to uuid as 1 argument'
...

```


However, currently I see that decimal have a bit different behaviour and 
raises always:

```

tarantool> decimal.new(1) == newproxy()
---
- error: '[string "return decimal.new(1) == newproxy()"]:1: expected 
decimal, number
     or string as 2 argument'
...

tarantool> decimal.new(1) > newproxy()
---
- error: '[string "return decimal.new(1) > newproxy()"]:1: expected 
decimal, number
     or string as 1 argument'
...

```


IMO, the first case is better because we can check that two lua built-in 
types

are equal but can't compare them.

```

tarantool> {} == 'string'
---
- false
...

tarantool> {} > 'string'
---
- error: '[string "return {} > ''string''"]:1: attempt to compare string 
with table'
...

```




^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-06  1:30       ` Oleg Babin via Tarantool-patches
@ 2021-08-06 13:00         ` Safin Timur via Tarantool-patches
  2021-08-06 17:24           ` Safin Timur via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06 13:00 UTC (permalink / raw)
  To: Oleg Babin, Vladislav Shpilevoy; +Cc: tarantool-patches


On 06.08.2021 4:30, Oleg Babin wrote:
> 
> On 06.08.2021 03:23, Safin Timur wrote:
>>
>> i.e. whenever we see not datetime or interva value datetime_cmp return 
>> nil, and all comparisons always return false. Please give me know if 
>> that's incorrect scenario and we should raise exception. This was the 
>> way Oleg Bain has requested it to behave. (return false for bogus data)
>>
> Hi! Seems you understood me wrong. It was only about equal comparison.
> 
> ```
> 
> tarantool> uuid.new() == newproxy()
> ---
> - false
> ...
> 
> tarantool> uuid.new() > newproxy()
> ---
> - error: '[string "return require(''uuid'').new() > newproxy()"]:1: 
> incorrect value
>      to convert to uuid as 1 argument'
> ...
> 
> ```
> 
> 
> However, currently I see that decimal have a bit different behaviour and 
> raises always:
> 
> ```
> 
> tarantool> decimal.new(1) == newproxy()
> ---
> - error: '[string "return decimal.new(1) == newproxy()"]:1: expected 
> decimal, number
>      or string as 2 argument'
> ...
> 
> tarantool> decimal.new(1) > newproxy()
> ---
> - error: '[string "return decimal.new(1) > newproxy()"]:1: expected 
> decimal, number
>      or string as 1 argument'
> ...
> 
> ```
> 
> 
> IMO, the first case is better because we can check that two lua built-in 
> types
> 
> are equal but can't compare them.
> 
> ```
> 
> tarantool> {} == 'string'
> ---
> - false
> ...
> 
> tarantool> {} > 'string'
> ---
> - error: '[string "return {} > ''string''"]:1: attempt to compare string 
> with table'
> ...
> 
> ```
> 
> 

Now, that looks more or less consistent, will do it such!

```
tarantool> date = require 'datetime'
---
...

tarantool> T = date('1970-01-01')
---
...

tarantool> T == false
---
- false
...

tarantool> T == true
---
- false
...

tarantool> T < false
---
- error: '[string "return T < false"]:1: incompatible types for comparison'
...

tarantool> T <= 'string'
---
- error: '[string "return T <= ''string''"]:1: incompatible types for 
comparison'
...
```

Patch is simple though
==============================================================
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
index 0bc86c9f3..509a2981f 100644
--- a/src/lua/datetime.lua
+++ b/src/lua/datetime.lua
@@ -245,12 +245,12 @@ end

  local function datetime_lt(lhs, rhs)
      local rc = datetime_cmp(lhs, rhs)
-    return rc ~= nil and rc < 0 or false
+    return rc ~= nil and rc < 0 or error('incompatible types for 
comparison', 2)
  end

  local function datetime_le(lhs, rhs)
      local rc = datetime_cmp(lhs, rhs)
-    return rc ~= nil and rc <= 0 or false
+    return rc ~= nil and rc <= 0 or  error('incompatible types for 
comparison', 2)
  end

  local function datetime_serialize(self)
==============================================================

Do we want to use that approach here?

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-06 13:00         ` Safin Timur via Tarantool-patches
@ 2021-08-06 17:24           ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-06 17:24 UTC (permalink / raw)
  To: Oleg Babin, Vladislav Shpilevoy; +Cc: tarantool-patches


On 06.08.2021 16:00, Safin Timur via Tarantool-patches wrote:
> 
> On 06.08.2021 4:30, Oleg Babin wrote:
>>
>> On 06.08.2021 03:23, Safin Timur wrote:
>>>
>>> i.e. whenever we see not datetime or interva value datetime_cmp 
>>> return nil, and all comparisons always return false. Please give me 
>>> know if that's incorrect scenario and we should raise exception. This 
>>> was the way Oleg Bain has requested it to behave. (return false for 
>>> bogus data)
>>>
>> Hi! Seems you understood me wrong. It was only about equal comparison.
>>
>> ```
>>
>> tarantool> uuid.new() == newproxy()
>> ---
>> - false
>> ...
>>
>> tarantool> uuid.new() > newproxy()
>> ---
>> - error: '[string "return require(''uuid'').new() > newproxy()"]:1: 
>> incorrect value
>>      to convert to uuid as 1 argument'
>> ...
>>
>> ```
>>
>>
>> However, currently I see that decimal have a bit different behaviour 
>> and raises always:
>>
>> ```
>>
>> tarantool> decimal.new(1) == newproxy()
>> ---
>> - error: '[string "return decimal.new(1) == newproxy()"]:1: expected 
>> decimal, number
>>      or string as 2 argument'
>> ...
>>
>> tarantool> decimal.new(1) > newproxy()
>> ---
>> - error: '[string "return decimal.new(1) > newproxy()"]:1: expected 
>> decimal, number
>>      or string as 1 argument'
>> ...
>>
>> ```
>>
>>
>> IMO, the first case is better because we can check that two lua 
>> built-in types
>>
>> are equal but can't compare them.
>>
>> ```
>>
>> tarantool> {} == 'string'
>> ---
>> - false
>> ...
>>
>> tarantool> {} > 'string'
>> ---
>> - error: '[string "return {} > ''string''"]:1: attempt to compare 
>> string with table'
>> ...
>>
>> ```
>>
>>
> 
> Now, that looks more or less consistent, will do it such!
> 
> ```
> tarantool> date = require 'datetime'
> ---
> ...
> 
> tarantool> T = date('1970-01-01')
> ---
> ...
> 
> tarantool> T == false
> ---
> - false
> ...
> 
> tarantool> T == true
> ---
> - false
> ...
> 
> tarantool> T < false
> ---
> - error: '[string "return T < false"]:1: incompatible types for comparison'
> ...
> 
> tarantool> T <= 'string'
> ---
> - error: '[string "return T <= ''string''"]:1: incompatible types for 
> comparison'
> ...
> ```
> 
> Patch is simple though
> ==============================================================
> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> index 0bc86c9f3..509a2981f 100644
> --- a/src/lua/datetime.lua
> +++ b/src/lua/datetime.lua
> @@ -245,12 +245,12 @@ end
> 
>   local function datetime_lt(lhs, rhs)
>       local rc = datetime_cmp(lhs, rhs)
> -    return rc ~= nil and rc < 0 or false
> +    return rc ~= nil and rc < 0 or error('incompatible types for 
> comparison', 2)
>   end
> 
>   local function datetime_le(lhs, rhs)
>       local rc = datetime_cmp(lhs, rhs)
> -    return rc ~= nil and rc <= 0 or false
> +    return rc ~= nil and rc <= 0 or  error('incompatible types for 
> comparison', 2)
>   end
> 
>   local function datetime_serialize(self)
> ==============================================================

Well, actually that was incorrect version (it was raising error if we 
have compatible types but comparison failed), condition should look 
different, but idea you saw correct:

============================================================
local function datetime_eq(lhs, rhs)
     local rc = datetime_cmp(lhs, rhs)
     return rc ~= nil and rc == 0
end

local function datetime_lt(lhs, rhs)
     local rc = datetime_cmp(lhs, rhs)
     return rc == nil and error('incompatible types for comparison', 2) or
            rc < 0
end

local function datetime_le(lhs, rhs)
     local rc = datetime_cmp(lhs, rhs)
     return rc == nil and error('incompatible types for comparison', 2) or
            rc <= 0
end
============================================================
> 
> Do we want to use that approach here?
> 
> Thanks,
> Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-06  0:23     ` Safin Timur via Tarantool-patches
  2021-08-06  1:30       ` Oleg Babin via Tarantool-patches
@ 2021-08-08 11:26       ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-08 16:35         ` Safin Timur via Tarantool-patches
  1 sibling, 1 reply; 38+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-08 11:26 UTC (permalink / raw)
  To: Safin Timur, olegrok; +Cc: tarantool-patches

Thanks for the fixes!

>>> +local datetime_t = ffi.typeof('struct datetime_t')
>>> +local interval_t = ffi.typeof('struct datetime_interval_t')
>>> +
>>> +local function is_interval(o)
>>> +    return type(o) == 'cdata' and ffi.istype(interval_t, o)
>>> +end
>>> +
>>> +local function is_datetime(o)
>>> +    return type(o) == 'cdata' and ffi.istype(datetime_t, o)
>>> +end
>>> +
>>> +local function is_date_interval(o)
>>> +    return is_datetime(o) or is_interval(o)
>>
>> 4. You make type(o) == 'cdata' check 2 times here.
> 
> Yes. It's not a problem, IMHO. LuaJIT after inlining of those 2 functions should take care of that. (I believe)

Please, remove the double-check. If you believe in something, it does not
necessarily is true, and sometimes things can't be jitted at all or not
as you expect. I would understand reliance on gcc, but LuaJIT is much
less reliable and sucks much more when something can't be jitted.

>> Do you have any tests? In this commit I see zero tests for
>> your code. Only very few tests for dt library. Please, add
>> tests for everything you have here.
> 
> A lot. It started to be added with the next commit, and added with any other commit adding functionality to datetime.lua. It's pity you didn't have a chance to see next commits.

No, I am really sorry, but this is not how it works. You need to send
atomic commits. Self-sufficient, covered with tests if possible. I can't
look at all your commits at once trying to understand what errors you
fixed in later commits and how.

>>> +        error(('value %d of %s is out of allowed range [%d, %d]'):
>>> +              format(v, txt, range[1], range[2]))
>>> +    end
>>> +end
>>> +
>>> +local datetime_index_handlers = {
>>> +    unixtime = function(self)
>>> +        return self.secs
>>> +    end,
>>> +
>>> +    timestamp = function(self)
>>> +        return tonumber(self.secs) + self.nsec / 1e9
>>
>> 11. If you are saying the Lua number value range is enough,
>> why do you store them as 64bit integers in 'self' and
>> convert to a number only for the external API?
> 
> May be I misunderstood your sentence, but let me elaborate here.
> I need seconds and nanoseconds kept separately for their efficient and precise handling, and for passing to c-dt.
> 
> If we would keep 32-bit integers in seconds then we would be able to handle only dates upto 2038 year, thus we need 64-bit seconds for extended range.
> 
> OTOH, not whole 64-bit range is practical, and required for expected in real-life datetime range values. It's not a problem that Lua number and int64_t have very different range for precise integer values. Lua number could handle integer values upto 9e15, which is corresponding to ...

I know all that. The question is why do you keep storing cdata 64 bits numbers
inside of the datetime objects, if you convert them all to Lua numbers before
return? You could store self.secs as just number. And the other fields too. Lua
number is double, it does not loose precision for integers < 2^53, which should
be enough for the ranges you want to support. Correct?

>>> +local function datetime_new(o)
>>> +    if o == nil then
>>> +        return datetime_new_raw(0, 0, 0)
>>> +    end
>>> +    local secs = 0
>>> +    local nsec = 0
>>> +    local offset = 0
>>> +    local easy_way = false
>>> +    local y = 0
>>> +    local M = 0
>>> +    local d = 0
>>> +    local ymd = false
>>> +
>>> +    local h = 0
>>> +    local m = 0
>>> +    local s = 0
>>> +    local frac = 0
>>> +    local hms = false
>>> +
>>> +    local dt = 0
>>> +
>>> +    local handlers = {
>>
>> 17. Please, stop using closures in the constructor of a
>> potentionally very frequently created/deleted object >
> 
> I still disagreed that datetime_new constructor will be on critical path, the most frequent case expected to be parse*() -> datetime_new_dt() -> datetime_new_raw(). i.e. when we parse millions of timestamps in logs and then proceed them.
> 
> datetime_new(builder_object) expected to be called in rare cases when object got created from Lua code, not on performance critical paths, thus convenience here is more important than performance.
> 
> And BTW I've created simple benchmark for comparing datetime_new_dt vs datetime_new(from builder object) vs datetime_new_2 which would be not using hash object with handlers, but rather sequence of ifs.
> 
> https://gist.github.com/tsafin/31cc9b0872b6015904fcc90d97740770
> ```
>  ~/datetime/tarantoolt [tsafin/gh-5941-datetime-v3]
> 00:39 $ ./build/src/tarantool ../bench-hash-dt.lua
> datetime_new_dt 0.002718448638916
> datetime_new    5.1318607330322
> datetime_new(precreated)        4.4448664188385
> datetime_new_2(ifs)     4.4977836608887
> ```
> 
> Note, that creation from builder object is much, much slower than raw cdata object creation (that's expected). If we move object out of loop, and precreate it then we save some time (as expected) 5.1 vs 4.4.
> But if we change implementation from hash with closure handlers to sequence of ifs - we actually degrade performance (a bit) from 4.44 to 4.49.
> 
> Biggest performance impact here - traversing of object attributes, but whether it's using closures or not using them, it makes no much difference.
> 
> But using closures makes this code look much more elegant. So keeping it here for today.

I want to see the version you created with ifs, please. With the closures
you do the same calculations + create the functions, it can't be slower
unless something is wrong with the version you did with ifs.

>> 19. It can ignore some fields (here it ignores 'year'):
>>
>> tarantool> d1 = datetime({secs = 100, year = 1})
>> ---
>> ...
>>
>> tarantool> d2 = datetime({secs = 100, year = 2})
>> ---
>> ...
>>
>> tarantool> d1 == d2
>> ---
>> - true
>> ...
>>
> 
> Yes, and that's by design. .secs defines 'easy_way' of initializing object via secs/nsec/offset, while .year selects 'ymd' mode. And `easy_way` has precedence. It looks reasonable for me, please, suggest your idea how to handle it even better without much redesign?

You could for instance, by looking at a first key you see in the dictionary,
decide how will you parse the entire dict. If you see first 'secs', you parse
only secs, nsecs and offset. Everything else raises an error. If you see
first 'year', you raise an error for the other non-related keys.

Ignorance of some keys does not look acceptable.

And by the way, does it mean I can't specify a date using year, month, day,
hours, and secs? Why isn't it valid?

>> 20. Serialization does not seem to be working in the console
>> despite you tried to implement it above:
>>
>> tarantool> d = datetime({secs = 100})
>> ---
>> ...
>>
>> tarantool> d
>> ---
>> - 'cdata<struct datetime_t>: 0x0103ad9c50'
>> ...
> 
> Yup, redefining of __serialize is not enough for cdata object by whatever reason (I didn't look deep). But after the next patch(es) it's start to stringize properly.

But don't you have __serialize implementation in this commit already?
(I don't remember and didn't see the latest code yet.)

>>> +            o = right_t == datetime_t and interval_new() or datetime_new()
>>> +            o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
>>> +                                            lhs.nsec - rhs.nsec)
>>> +            return o
>>> +        else
>>> +            error_incompatible("operator -")
>>> +        end
>>> +    -- both left and right are generic intervals
>>> +    elseif left_t == interval_t and right_t == interval_t then
>>> +        o = interval_new()
>>> +        o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
>>> +                                        lhs.nsec - rhs.nsec)
>>> +        return o
>>> +    else
>>> +        error_incompatible("operator -")
>>> +    end
>>> +end
>>
>> 22. Did you really test this function?
>>
>> tarantool> dt = require('datetime')
>> ---
>> ...
>>
>> tarantool> d1 = dt()
>> ---
>> ...
>>
>> tarantool> d2 - 1
>> ---
>> - error: 'builtin/datetime.lua:487: bad argument #1 to ''typeof'' (C type expected,
>>      got number)'
>> ...
> 
> All kinds of arguments checking implemented in the separate patch about interval arithmetic operations, please find a moment to look into them.

Please, do not introduce broken functions. Make the correct ones right away and
add tests. The new code does not need to introduce full functionality if it
can be done in an atomic way, but it must not be visibly broken at least. Here
it is.

>>> +
>>> +local function datetime_from(o)
>>> +    if o == nil or type(o) == 'table' then
>>> +        return datetime_new(o)
>>> +    elseif type(o) == 'string' then
>>> +        return parse(o)
>>> +    end
>>> +end
>>> +
>>> +local function local_now()
>>> +    local p_tv = ffi.new('struct timeval [1]')
>>> +    local rc = builtin.gettimeofday(p_tv, nil)
>>
>> 24. Why can't you use fiber time? It should be enough in
>> nearly all usecases and is much faster. For real-real time
>> talking directly to the kernel we have 'clock' Lua module
>> already. You could check if results of clock functions can
>> be passed into your module to create 'real' timestamps not
>> depending on the eventloop.
> 
> Fiber time, which actually return cached ev_rt_now, looks imprecise and depends on event loop circumstances, though eventually it calls the same clock_gettime(). So fiber-time is looking like a cached timestamp value from [hopefully, non-distant] past.
> 
> I'd rather prefer to have more realtime values, and clock_realtime*/clock_monotonic* look relevant, they call clock_gettime() for their purposes, which in turn calls the same gettimeofday(). So in this case we call same api, but with 2 extra call layers above.

By using gettimeofday you drag a lot of cdef garbage including the function
itself and what it returns. Please, don't. Anyway, the standard says struct
timeval can have more members than the usually used seconds and microseconds:

	https://pubs.opengroup.org/onlinepubs/7908799/xsh/systime.h.html

	The <sys/time.h> header defines the timeval structure that includes
	**at least** the following members

> IMHO, not worth extra complexity if we could just simply call gettimeofday().

Now you added a lot of extra complexity exactly because of this function.

>> 25. Functions like ctime can't be used in Lua, because according
>> to their Linux man:
>>
>>     The return value points to a statically allocated string
>>     which might be overwritten by subsequent calls to any of the
>>     date and time functions
>>
>> Which means they use a global buffer. If you will
>> use datetime module in any of __gc handlers, it will
>> return garbage sometimes or just override results
>> somethere. That is exactly the same problem as existed
>> with tarantool's static buffer.
> 
> I do not believe it's the same problem - in this case content of static buffer is immediately interned in Lua via ffi.string()

Well, it is exactly the same problem. How do you think the static
buffer used to be copied to Lua until we purged it from everywhere? Using
the same ffi.string(). Before ffi.string() will copy anything, it
will start GC on demand, and inside of GC the string might be overwritten.
So when GC ends, ffi.string() will copy garbage or some new foreign string.

> https://luajit.org/ext_ffi_api.html
> "The Lua string is an (interned) copy of the data and bears no relation to the original data area anymore."

It does not say anything about when GC is started, and that was the
problem with already 2 global buffers - our static one and one from
luacjson library. I don't see how a third global buffer is different.
It is broken in the same way precisely.

Look at this commit: acf8745ed8fef47e6d1f1c31708c7c9d6324d2f3
("uuid: drop tt_uuid_str() from Lua"). It is your case exactly.
A global buffer is returned to Lua via ffi and then ffi.string()
was called. There is a test which breaks when you do so. Strings
become wrong and/or garbage.

>>> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
>>> new file mode 100644
>>> index 000000000..b6f568c03
>>> --- /dev/null
>>> +++ b/test/unit/datetime.c
>>
>> 26. This file tests dt library. It does not test your code. Please, add
>> tests for everything you implemented in this commit. For c-dt library
>> the tests must be added in the previous commit, and the test file
>> must be called test/unit/cdt.c then.
> 
> Unit-test has been moved to the commit introducing c-dt. And in turn, initial datetime test (checking parser and ctime/asctime/now) squashed to the same commit where we introduce module with parser.

Thanks, that should be better.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime Timur Safin via Tarantool-patches
  2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-06  0:26   ` Safin Timur via Tarantool-patches
@ 2021-08-08 14:34   ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-08 16:47     ` Safin Timur via Tarantool-patches
  2 siblings, 1 reply; 38+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-08 14:34 UTC (permalink / raw)
  To: Timur Safin, olegrok; +Cc: tarantool-patches

Thanks for the patch!

See 7 comments below.

> diff --git a/src/exports.h b/src/exports.h
> index 5bb3e6a2b..fb4a8084f 100644
> --- a/src/exports.h
> +++ b/src/exports.h
> @@ -217,7 +217,32 @@ EXPORT(curl_url_set)
>  EXPORT(curl_version)
>  EXPORT(curl_version_info)
>  #endif /* EXPORT_LIBCURL_SYMBOLS */
> +EXPORT(datetime_asctime)
> +EXPORT(datetime_ctime)
> +EXPORT(datetime_now)
> +EXPORT(datetime_strftime)
>  EXPORT(decimal_unpack)
> +EXPORT(dt_dow)
> +EXPORT(dt_from_rdn)
> +EXPORT(dt_from_struct_tm)
> +EXPORT(dt_from_yd)
> +EXPORT(dt_from_ymd)
> +EXPORT(dt_from_yqd)
> +EXPORT(dt_from_ywd)
> +EXPORT(dt_parse_iso_date)
> +EXPORT(dt_parse_iso_time_basic)
> +EXPORT(dt_parse_iso_time_extended)
> +EXPORT(dt_parse_iso_time)
> +EXPORT(dt_parse_iso_zone_basic)
> +EXPORT(dt_parse_iso_zone_extended)
> +EXPORT(dt_parse_iso_zone_lenient)
> +EXPORT(dt_parse_iso_zone)
> +EXPORT(dt_rdn)
> +EXPORT(dt_to_struct_tm)
> +EXPORT(dt_to_yd)
> +EXPORT(dt_to_ymd)
> +EXPORT(dt_to_yqd)
> +EXPORT(dt_to_ywd)

1. In scope of https://github.com/tarantool/tarantool/pull/6265 the exported
symbols are heavily reworked. In particular, you should not export library
symbols as is, you need to wrap them into tt_cdt_* functions and export them
instead. Otherwise the users' modules, which link with their own version of
c-dt, will have conflicts.

>  EXPORT(error_ref)
>  EXPORT(error_set_prev)
>  EXPORT(error_unpack_unsafe)
> diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
> new file mode 100644
> index 000000000..e77b188b7
> --- /dev/null
> +++ b/src/lib/core/datetime.c
> @@ -0,0 +1,96 @@
> +/*
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * 1. Redistributions of source code must retain the above
> + *    copyright notice, this list of conditions and the
> + *    following disclaimer.
> + *
> + * 2. Redistributions in binary form must reproduce the above
> + *    copyright notice, this list of conditions and the following
> + *    disclaimer in the documentation and/or other materials
> + *    provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
> + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
> + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
> + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
> + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +
> +#include <string.h>
> +#include <time.h>
> +
> +#include "trivia/util.h"
> +#include "datetime.h"
> +
> +static int
> +local_dt(int64_t secs)
> +{
> +	return dt_from_rdn((int)(secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET);
> +}
> +
> +static struct tm *
> +datetime_to_tm(const struct datetime *date)
> +{
> +	static struct tm tm;
> +
> +	memset(&tm, 0, sizeof(tm));

2. Why do you need this memset if you overwrite all the
fields anyway?

> +	int64_t secs = date->secs;
> +	dt_to_struct_tm(local_dt(secs), &tm);
> +
> +	int seconds_of_day = date->secs % SECS_PER_DAY;
> +	tm.tm_hour = (seconds_of_day / 3600) % 24;
> +	tm.tm_min = (seconds_of_day / 60) % 60;
> +	tm.tm_sec = seconds_of_day % 60;
> +
> +	return &tm;
> +}
> +
> +void
> +datetime_now(struct datetime * now)

3. Please, do not use whitespace between * and the variable name.

> +{
> +	struct timeval tv;
> +	gettimeofday(&tv, NULL);
> +	now->secs = tv.tv_sec;
> +	now->nsec = tv.tv_usec * 1000;
> +
> +	time_t now_seconds;
> +	time(&now_seconds);
> +	struct tm tm;
> +	localtime_r(&now_seconds, &tm);

4. Wtf? 2 system calls just to get the current time? Can you use
just one? Why can't you use tv.tv_sec which you got above?

> +	now->offset = tm.tm_gmtoff / 60;
> +}
> +
> +char *
> +datetime_asctime(const struct datetime *date)
> +{
> +	struct tm *p_tm = datetime_to_tm(date);
> +	return asctime(p_tm);

5. You can't return a global buffer. You need to write the
result into a local one passed in the arguments. AFAIS, there
are functions with _r suffix which should help.

> +}
> +
> +char *
> +datetime_ctime(const struct datetime *date)
> +{
> +	time_t time = date->secs;
> +	return ctime(&time);
> +}
> +
> +size_t
> +datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +		  uint32_t len)
> +{
> +	struct tm *p_tm = datetime_to_tm(date);
> +	return strftime(buf, len, fmt, p_tm);
> +}
> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> new file mode 100644
> index 000000000..6699b31d3
> --- /dev/null
> +++ b/src/lib/core/datetime.h
> @@ -0,0 +1,94 @@
> +#pragma once
> +/*
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + *
> + * Redistribution and use in source and binary forms, with or
> + * without modification, are permitted provided that the following
> + * conditions are met:
> + *
> + * 1. Redistributions of source code must retain the above
> + *    copyright notice, this list of conditions and the
> + *    following disclaimer.
> + *
> + * 2. Redistributions in binary form must reproduce the above
> + *    copyright notice, this list of conditions and the following
> + *    disclaimer in the documentation and/or other materials
> + *    provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
> + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
> + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
> + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
> + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include <stdio.h>
> +#include "c-dt/dt.h"
> +
> +#if defined(__cplusplus)
> +extern "C"
> +{
> +#endif /* defined(__cplusplus) */
> +
> +#ifndef SECS_PER_DAY
> +#define SECS_PER_DAY	86400

6. You have a tab here after the name. Please, use whitespace.

> +#define DT_EPOCH_1970_OFFSET 719163
> +#endif
> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> new file mode 100644
> index 000000000..a562b6872
> --- /dev/null
> +++ b/src/lua/datetime.lua
> @@ -0,0 +1,638 @@
> +local ffi = require('ffi')
> +
> +ffi.cdef [[
> +
> +    /*
> +    `c-dt` library functions handles properly both positive and negative `dt`
> +    values, where `dt` is a number of dates since Rata Die date (0001-01-01).
> +
> +    For better compactness of our typical data in MessagePack stream we shift
> +    root of our time to the Unix Epoch date (1970-01-01), thus our 0 is
> +    actually dt = 719163.
> +
> +    So here is a simple formula how convert our epoch-based seconds to dt values
> +        dt = (secs / 86400) + 719163
> +    Where 719163 is an offset of Unix Epoch (1970-01-01) since Rata Die
> +    (0001-01-01) in dates.
> +
> +    */
> +    typedef int dt_t;
> +
> +    // dt_core.h
> +    typedef enum {
> +        DT_MON       = 1,
> +        DT_MONDAY    = 1,
> +        DT_TUE       = 2,
> +        DT_TUESDAY   = 2,
> +        DT_WED       = 3,
> +        DT_WEDNESDAY = 3,
> +        DT_THU       = 4,
> +        DT_THURSDAY  = 4,
> +        DT_FRI       = 5,
> +        DT_FRIDAY    = 5,
> +        DT_SAT       = 6,
> +        DT_SATURDAY  = 6,
> +        DT_SUN       = 7,
> +        DT_SUNDAY    = 7,
> +    } dt_dow_t;
> +
> +    dt_t     dt_from_rdn     (int n);
> +    dt_t     dt_from_yd      (int y, int d);
> +    dt_t     dt_from_ymd     (int y, int m, int d);
> +    dt_t     dt_from_yqd     (int y, int q, int d);
> +    dt_t     dt_from_ywd     (int y, int w, int d);
> +
> +    void     dt_to_yd        (dt_t dt, int *y, int *d);
> +    void     dt_to_ymd       (dt_t dt, int *y, int *m, int *d);
> +    void     dt_to_yqd       (dt_t dt, int *y, int *q, int *d);
> +    void     dt_to_ywd       (dt_t dt, int *y, int *w, int *d);
> +
> +    int      dt_rdn          (dt_t dt);
> +    dt_dow_t dt_dow          (dt_t dt);

7. dt_dow is not used in this file. The enum dt_dow_t either.
The same for dt_from_rdn, dt_from_yd, dt_from_yqd, dt_from_ywd,
dt_to_yd, dt_to_ymd, dt_to_yqd, dt_to_ywd. Please, check all the
functions.

I will return later with more comments. Need to go now.


^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build
  2021-08-05  8:55     ` Safin Timur via Tarantool-patches
@ 2021-08-08 14:34       ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-08 14:34 UTC (permalink / raw)
  To: Safin Timur, olegrok; +Cc: tarantool-patches

On 05.08.2021 11:55, Safin Timur wrote:
> Hello Vlad!
> 
> On 05.08.2021 2:58, Vladislav Shpilevoy wrote:
>> Hi! Thanks for the patch!
>>
>> On 02.08.2021 02:40, Timur Safin via Tarantool-patches wrote:
>>> * Integrated chansen/c-dt parser as 3rd party module to the
>>>    Tarantool cmake build process.
>>> * Points to tsafin/c-dt instead iof original chansen/c-dt to
>>>    have easier build integration, because there is additional
>>>    commit which integrated cmake support
>>
>> As I said in the previous review, it points at
>> https://github.com/tarantool/c-dt.git, not at tsafin/c-dt. Why
>> do you keep saying the contrary? Look:
>>
>>     url = https://github.com/tarantool/c-dt.git
>>
>> It is 'tarantool', not 'tsafin' here.
> 
> Yes, that was oversight, due to several restarts of rebase session
> (I guess due to mess generated with combination of --autosquash and git rerere)
> Now finally applied to this commit in branch as commit #4b90de3bb21b9f28bc7c1f8e851d4c95f1b0f191.
> 
>>
>> Also, like on the previous review, when I call 'make' and do
>> 'git status', I see:
>>
>>     Changes not staged for commit:
>>       (use "git add <file>..." to update what will be committed)
>>       (use "git restore <file>..." to discard changes in working directory)
>>       (commit or discard the untracked or modified content in submodules)
>>         modified:   third_party/c-dt (modified content, untracked content)
>>
>> You still didn't fix it. Please, do.
>>
> 
> That was not a problem for me, due to the fact I _always_ use out-of-source build, and all artifacts generated separately under ${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt/, and such artifacts
> inside of source tree problem observed only when you do (not very idiomatic) in-source build. cmake generated Makefile overwrite original c-dt Makefile.
> 
> [While we are here, could you please remind me why we recommend to use in-source build in our documentation?
> And not prefer the recommended and used by majority of industry for what matter out-of-source build?]

No idea. I personally use in-source build because I have just used to it.

> We could rather may not influence how and where artifacts will be generated from inside of our own c-dt/CMakeLists.txt, we should rather modify the way how it get used, like, not include it via simple add_subdirectory(which is exactly defining this ${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt, which would be equal to ${CMAKE_CURRENT_SOURCE_DIR}/third_party/c-dt for in-source build, but rather use some ExternalProject_Add() techniques, which allows to redefine generated and binary files localtion. Let me check what is the easiest way here...

I am not a cmake master, so I don't know a correct way to fix it. But I
don't understand why do you need Makefile in c-dt at all if it works via
cmake anyway and 100% of that Makefile is simply overwritten. It is not
used. If a file is not used and is overwritten anyway, why not drop it
from the repository?

Also I see

	Untracked files:
	  (use "git add <file>..." to include in what will be committed)
		libcdt.a

You need to update gitignore so as there is no the warning.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-08 11:26       ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-08-08 16:35         ` Safin Timur via Tarantool-patches
  2021-08-10 12:20           ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-08 16:35 UTC (permalink / raw)
  To: Vladislav Shpilevoy, olegrok; +Cc: tarantool-patches

Much respect that you've found some time for review, even while being on 
vacation! Thanks!

On 08.08.2021 14:26, Vladislav Shpilevoy wrote:
>>>> +local datetime_index_handlers = {
>>>> +    unixtime = function(self)
>>>> +        return self.secs
>>>> +    end,
>>>> +
>>>> +    timestamp = function(self)
>>>> +        return tonumber(self.secs) + self.nsec / 1e9
>>>
>>> 11. If you are saying the Lua number value range is enough,
>>> why do you store them as 64bit integers in 'self' and
>>> convert to a number only for the external API?
>>
>> May be I misunderstood your sentence, but let me elaborate here.
>> I need seconds and nanoseconds kept separately for their efficient and precise handling, and for passing to c-dt.
>>
>> If we would keep 32-bit integers in seconds then we would be able to handle only dates upto 2038 year, thus we need 64-bit seconds for extended range.
>>
>> OTOH, not whole 64-bit range is practical, and required for expected in real-life datetime range values. It's not a problem that Lua number and int64_t have very different range for precise integer values. Lua number could handle integer values upto 9e15, which is corresponding to ...
> 
> I know all that. The question is why do you keep storing cdata 64 bits numbers
> inside of the datetime objects, if you convert them all to Lua numbers before
> return? You could store self.secs as just number. And the other fields too. Lua
> number is double, it does not loose precision for integers < 2^53, which should
> be enough for the ranges you want to support. Correct?

I keep using 64-bit because the primary code operating with fields is on 
C-side, and we need Lua number only on rare cases when user asked for 
composed attribute date.timestamp. Whenever we deal with seconds within 
arthmetic operations or transfer to c-dt, we need integer C type, larger 
than 32-bit. It's good fit for int64_t.


> 
>>>> +local function datetime_new(o)
>>>> +    if o == nil then
>>>> +        return datetime_new_raw(0, 0, 0)
>>>> +    end
>>>> +    local secs = 0
>>>> +    local nsec = 0
>>>> +    local offset = 0
>>>> +    local easy_way = false
>>>> +    local y = 0
>>>> +    local M = 0
>>>> +    local d = 0
>>>> +    local ymd = false
>>>> +
>>>> +    local h = 0
>>>> +    local m = 0
>>>> +    local s = 0
>>>> +    local frac = 0
>>>> +    local hms = false
>>>> +
>>>> +    local dt = 0
>>>> +
>>>> +    local handlers = {
>>>
>>> 17. Please, stop using closures in the constructor of a
>>> potentionally very frequently created/deleted object >
>>
>> I still disagreed that datetime_new constructor will be on critical path, the most frequent case expected to be parse*() -> datetime_new_dt() -> datetime_new_raw(). i.e. when we parse millions of timestamps in logs and then proceed them.
>>
>> datetime_new(builder_object) expected to be called in rare cases when object got created from Lua code, not on performance critical paths, thus convenience here is more important than performance.
>>
>> And BTW I've created simple benchmark for comparing datetime_new_dt vs datetime_new(from builder object) vs datetime_new_2 which would be not using hash object with handlers, but rather sequence of ifs.
>>
>> https://gist.github.com/tsafin/31cc9b0872b6015904fcc90d97740770
>> ```
>>   ~/datetime/tarantoolt [tsafin/gh-5941-datetime-v3]
>> 00:39 $ ./build/src/tarantool ../bench-hash-dt.lua
>> datetime_new_dt 0.002718448638916
>> datetime_new    5.1318607330322
>> datetime_new(precreated)        4.4448664188385
>> datetime_new_2(ifs)     4.4977836608887
>> ```
>>
>> Note, that creation from builder object is much, much slower than raw cdata object creation (that's expected). If we move object out of loop, and precreate it then we save some time (as expected) 5.1 vs 4.4.
>> But if we change implementation from hash with closure handlers to sequence of ifs - we actually degrade performance (a bit) from 4.44 to 4.49.
>>
>> Biggest performance impact here - traversing of object attributes, but whether it's using closures or not using them, it makes no much difference.
>>
>> But using closures makes this code look much more elegant. So keeping it here for today.
> 
> I want to see the version you created with ifs, please. With the closures
> you do the same calculations + create the functions, it can't be slower
> unless something is wrong with the version you did with ifs.

I've provided you the link to this benchmark - please see gist 
https://gist.github.com/tsafin/31cc9b0872b6015904fcc90d97740770

But now looked into this code again, and discovered that I didn't call 
datetime_new_2 (version with ifs) in the last case, but rather 
copy-pasted datetime_new (original version), and timings should be 
actually the same as in precreated version, where we created object with 
attributes before loop, and then used it as invariant.

So here are local timings with corrected code. Ran 3 times:

```
✔ ~/datetime/tarantoolt [tsafin/gh-5941-datetime-v3|…1⚑ 3]
18:01 $ ./build/src/tarantool ../bench-hash-dt.lua
datetime_new_dt 0.001617431640625
datetime_new    3.0791389942169
datetime_new(precreated)        3.1896848678589
datetime_new_2(ifs)     0.15672922134399
✔ ~/datetime/tarantoolt [tsafin/gh-5941-datetime-v3|…1⚑ 3]
18:02 $ ./build/src/tarantool ../bench-hash-dt.lua
datetime_new_dt 0.0017032623291016
datetime_new    3.0739498138428
datetime_new(precreated)        2.9590804576874
datetime_new_2(ifs)     0.14570164680481
✔ ~/datetime/tarantoolt [tsafin/gh-5941-datetime-v3|…1⚑ 3]
18:12 $ ./build/src/tarantool ../bench-hash-dt.lua
datetime_new_dt 0.0016193389892578
datetime_new    3.4707424640656
datetime_new(precreated)        3.1298229694366
datetime_new_2(ifs)     0.14686107635498
```

So yes, you were correct, version with ifs is 20x times faster than more 
readable version using hash with closures.


> 
>>> 19. It can ignore some fields (here it ignores 'year'):
>>>
>>> tarantool> d1 = datetime({secs = 100, year = 1})
>>> ---
>>> ...
>>>
>>> tarantool> d2 = datetime({secs = 100, year = 2})
>>> ---
>>> ...
>>>
>>> tarantool> d1 == d2
>>> ---
>>> - true
>>> ...
>>>
>>
>> Yes, and that's by design. .secs defines 'easy_way' of initializing object via secs/nsec/offset, while .year selects 'ymd' mode. And `easy_way` has precedence. It looks reasonable for me, please, suggest your idea how to handle it even better without much redesign?
> 
> You could for instance, by looking at a first key you see in the dictionary,
> decide how will you parse the entire dict. If you see first 'secs', you parse
> only secs, nsecs and offset. Everything else raises an error. If you see
> first 'year', you raise an error for the other non-related keys.
> 
> Ignorance of some keys does not look acceptable.
> 
> And by the way, does it mean I can't specify a date using year, month, day,
> hours, and secs? Why isn't it valid?

Here is the tricky part, agreed that this may be confusing. The "easy 
mode" operates on secs, nsec, offset, the human-readable mode operates 
on year, month, day, hour, minute, **second**, and tz.

There is second in human-readable mode, it's rather named differently 
(i.e. complete word, non-plural).

The more important question though - how to implement reasonably 
diagnosing of such mix-in of raw attributes, with human-radable attributes?

- I tend to believe that we should **not accept them both** in the 
single function, but rather accept .secs/.nsec/.offset via different 
entries, like datetime_new_raw vs datetime_new

```
date = require('datetime')
T = date.new{year = 1970, month = 1, day = 1, minute = 0, seconds = 15, 
tz = 180}
T2 = date.new_raw{secs = 0, nsec = 0, offset = 180} -- will be the same 
as new(0, 0, 180)
```

In this case we physically prevent mixing of different modes with 
different allowed attributes in the builder object.

What do you think?


> 
>>> 20. Serialization does not seem to be working in the console
>>> despite you tried to implement it above:
>>>
>>> tarantool> d = datetime({secs = 100})
>>> ---
>>> ...
>>>
>>> tarantool> d
>>> ---
>>> - 'cdata<struct datetime_t>: 0x0103ad9c50'
>>> ...
>>
>> Yup, redefining of __serialize is not enough for cdata object by whatever reason (I didn't look deep). But after the next patch(es) it's start to stringize properly.
> 
> But don't you have __serialize implementation in this commit already?
> (I don't remember and didn't see the latest code yet.)

Yes, I did have __serialize there, but that was working strangely and 
only started to behave properly (well, more or less) once I've added 
__tostring in the following commit. I'm not very well informed about 
expected console behaviour (regarding those 2 meta-methods, at least), 
so took that for granted. Probably, there is some bug, may be it's 
expected behaviour for cdata based data structures?


> 
>>>> +            o = right_t == datetime_t and interval_new() or datetime_new()
>>>> +            o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
>>>> +                                            lhs.nsec - rhs.nsec)
>>>> +            return o
>>>> +        else
>>>> +            error_incompatible("operator -")
>>>> +        end
>>>> +    -- both left and right are generic intervals
>>>> +    elseif left_t == interval_t and right_t == interval_t then
>>>> +        o = interval_new()
>>>> +        o.secs, o.nsec = _normalize_nsec(lhs.secs - rhs.secs,
>>>> +                                        lhs.nsec - rhs.nsec)
>>>> +        return o
>>>> +    else
>>>> +        error_incompatible("operator -")
>>>> +    end
>>>> +end
>>>
>>> 22. Did you really test this function?
>>>
>>> tarantool> dt = require('datetime')
>>> ---
>>> ...
>>>
>>> tarantool> d1 = dt()
>>> ---
>>> ...
>>>
>>> tarantool> d2 - 1
>>> ---
>>> - error: 'builtin/datetime.lua:487: bad argument #1 to ''typeof'' (C type expected,
>>>       got number)'
>>> ...
>>
>> All kinds of arguments checking implemented in the separate patch about interval arithmetic operations, please find a moment to look into them.
> 
> Please, do not introduce broken functions. Make the correct ones right away and
> add tests. The new code does not need to introduce full functionality if it
> can be done in an atomic way, but it must not be visibly broken at least. Here
> it is.

Ok, here is how it's looking in patchset: in 1 commit we introduce 
arithmetic support, but not handle well all corner cases (especially 
there is no support for proper year or month arithmetics). Special types 
for year, and months date intervals have been added in the next commit, 
where we handle all corner cases with bogus data.

We could:
- either squash interval support to the original module;
- or remove arithmetic altogether from original patch creating the 
module datetime.lua.


What is the recommended way?

> 
>>>> +
>>>> +local function datetime_from(o)
>>>> +    if o == nil or type(o) == 'table' then
>>>> +        return datetime_new(o)
>>>> +    elseif type(o) == 'string' then
>>>> +        return parse(o)
>>>> +    end
>>>> +end
>>>> +
>>>> +local function local_now()
>>>> +    local p_tv = ffi.new('struct timeval [1]')
>>>> +    local rc = builtin.gettimeofday(p_tv, nil)
>>>
>>> 24. Why can't you use fiber time? It should be enough in
>>> nearly all usecases and is much faster. For real-real time
>>> talking directly to the kernel we have 'clock' Lua module
>>> already. You could check if results of clock functions can
>>> be passed into your module to create 'real' timestamps not
>>> depending on the eventloop.
>>
>> Fiber time, which actually return cached ev_rt_now, looks imprecise and depends on event loop circumstances, though eventually it calls the same clock_gettime(). So fiber-time is looking like a cached timestamp value from [hopefully, non-distant] past.
>>
>> I'd rather prefer to have more realtime values, and clock_realtime*/clock_monotonic* look relevant, they call clock_gettime() for their purposes, which in turn calls the same gettimeofday(). So in this case we call same api, but with 2 extra call layers above.
> 
> By using gettimeofday you drag a lot of cdef garbage including the function
> itself and what it returns. Please, don't. Anyway, the standard says struct
> timeval can have more members than the usually used seconds and microseconds:
> 
> 	https://pubs.opengroup.org/onlinepubs/7908799/xsh/systime.h.html
> 
> 	The <sys/time.h> header defines the timeval structure that includes
> 	**at least** the following members
> 
>> IMHO, not worth extra complexity if we could just simply call gettimeofday().
> 
> Now you added a lot of extra complexity exactly because of this function.

Once again, this code is in C now, there is no any non-portable timeval 
definition of ffi to gettimeofday in Lua - everything is in C, which 
extra complexity you are talking about? I do not understand.


> 
>>> 25. Functions like ctime can't be used in Lua, because according
>>> to their Linux man:
>>>
>>>      The return value points to a statically allocated string
>>>      which might be overwritten by subsequent calls to any of the
>>>      date and time functions
>>>
>>> Which means they use a global buffer. If you will
>>> use datetime module in any of __gc handlers, it will
>>> return garbage sometimes or just override results
>>> somethere. That is exactly the same problem as existed
>>> with tarantool's static buffer.
>>
>> I do not believe it's the same problem - in this case content of static buffer is immediately interned in Lua via ffi.string()
> 
> Well, it is exactly the same problem. How do you think the static
> buffer used to be copied to Lua until we purged it from everywhere? Using
> the same ffi.string(). Before ffi.string() will copy anything, it
> will start GC on demand, and inside of GC the string might be overwritten.
> So when GC ends, ffi.string() will copy garbage or some new foreign string.
> 
>> https://luajit.org/ext_ffi_api.html
>> "The Lua string is an (interned) copy of the data and bears no relation to the original data area anymore."
> 
> It does not say anything about when GC is started, and that was the
> problem with already 2 global buffers - our static one and one from
> luacjson library. I don't see how a third global buffer is different.
> It is broken in the same way precisely.
> 
> Look at this commit: acf8745ed8fef47e6d1f1c31708c7c9d6324d2f3
> ("uuid: drop tt_uuid_str() from Lua"). It is your case exactly.
> A global buffer is returned to Lua via ffi and then ffi.string()
> was called. There is a test which breaks when you do so. Strings
> become wrong and/or garbage.

Thanks for the hint, that's very nice test, and quite convincing! I've 
added datetime.ctime and datetime.asctime checkes there:
==========================================================
diff --git a/test/app-tap/gh-5632-6050-6259-gc-buf-reuse.test.lua 
b/test/app-tap/gh-5632-6050-6259-gc-buf-reuse.test.lua
index 84c2944e5..7eb05f9e8 100755
--- a/test/app-tap/gh-5632-6050-6259-gc-buf-reuse.test.lua
+++ b/test/app-tap/gh-5632-6050-6259-gc-buf-reuse.test.lua
@@ -14,6 +14,7 @@ local uri = require('uri')
  local json = require('json')
  local msgpackffi = require('msgpackffi')
  local decimal = require('decimal')
+local datetime = require('datetime')

  local function test_uuid(test)
      test:plan(1)
@@ -256,15 +257,86 @@ local function test_decimal(test)
      test:ok(is_success, 'decimal str in gc')
  end

+local function test_datetime_asctime(test)
+    test:plan(1)
+
+    local gc_count = 100
+    local iter_count = 1000
+    local is_success = true
+
+    local T1 = datetime('1970-01-01')
+    local T2 = datetime('2000-01-01')
+
+    local function datetime_asctime()
+        local str1 = datetime.asctime(T1)
+        local str2 = datetime.asctime(T2)
+        local str3 = datetime.asctime(T1)
+        local str4 = datetime.asctime(T2)
+        if str1 ~= str3 or str2 ~= str4 then
+            is_success = false
+        end
+    end
+
+    local function create_gc()
+        for _ = 1, gc_count do
+            ffi.gc(ffi.new('char[1]'), function() datetime_asctime() end)
+        end
+    end
+
+    for _ = 1, iter_count do
+        create_gc()
+        datetime_asctime()
+    end
+
+    test:ok(is_success, 'info datetime in gc')
+end
+
+local function test_datetime_ctime(test)
+    test:plan(1)
+
+    local gc_count = 100
+    local iter_count = 1000
+    local is_success = true
+
+    local T1 = datetime('1970-01-01')
+    local T2 = datetime('2000-01-01')
+
+    local function datetime_ctime()
+        local str1 = datetime.ctime(T1)
+        local str2 = datetime.ctime(T2)
+        local str3 = datetime.ctime(T1)
+        local str4 = datetime.ctime(T2)
+        if str1 ~= str3 or str2 ~= str4 then
+            is_success = false
+        end
+    end
+
+    local function create_gc()
+        for _ = 1, gc_count do
+            ffi.gc(ffi.new('char[1]'), function() datetime_ctime() end)
+        end
+    end
+
+    for _ = 1, iter_count do
+        create_gc()
+        datetime_ctime()
+    end
+
+    test:ok(is_success, 'info datetime in gc')
+end
+
+
  box.cfg{}

  local test = tap.test('gh-5632-6050-6259-gc-buf-reuse')
-test:plan(6)
+test:plan(8)
  test:test('uuid in __gc', test_uuid)
  test:test('uri in __gc', test_uri)
  test:test('msgpackffi in __gc', test_msgpackffi)
  test:test('json in __gc', test_json)
  test:test('info uuid in __gc', test_info_uuid)
  test:test('decimal str in __gc', test_decimal)
+test:test('datetime.asctime in __gc', test_datetime_asctime)
+test:test('datetime.ctime in __gc', test_datetime_ctime)

  os.exit(test:check() and 0 or 1)
==========================================================

And regardless of a number of times I ran it in Debug mode - it was 
never failing. But...

Once, I've recompiled branch in Release mode - it start to faile evrry 
next moment. I'll play with reentrant versions of ctime (ctime_r) and 
asctime (asctime_r), making sure that LuaJIT allocators will know about 
buffers passed to and returned from those functions.

> 
>>>> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
>>>> new file mode 100644
>>>> index 000000000..b6f568c03
>>>> --- /dev/null
>>>> +++ b/test/unit/datetime.c
>>>
>>> 26. This file tests dt library. It does not test your code. Please, add
>>> tests for everything you implemented in this commit. For c-dt library
>>> the tests must be added in the previous commit, and the test file
>>> must be called test/unit/cdt.c then.
>>
>> Unit-test has been moved to the commit introducing c-dt. And in turn, initial datetime test (checking parser and ctime/asctime/now) squashed to the same commit where we introduce module with parser.
> 
> Thanks, that should be better.
> 

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-08 14:34   ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-08-08 16:47     ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-08 16:47 UTC (permalink / raw)
  To: Vladislav Shpilevoy, olegrok; +Cc: tarantool-patches



On 08.08.2021 17:34, Vladislav Shpilevoy wrote:
> Thanks for the patch!
> 
> See 7 comments below.
> 
>> diff --git a/src/exports.h b/src/exports.h
>> index 5bb3e6a2b..fb4a8084f 100644
>> --- a/src/exports.h
>> +++ b/src/exports.h
>> @@ -217,7 +217,32 @@ EXPORT(curl_url_set)
>>   EXPORT(curl_version)
>>   EXPORT(curl_version_info)
>>   #endif /* EXPORT_LIBCURL_SYMBOLS */
>> +EXPORT(datetime_asctime)
>> +EXPORT(datetime_ctime)
>> +EXPORT(datetime_now)
>> +EXPORT(datetime_strftime)
>>   EXPORT(decimal_unpack)
>> +EXPORT(dt_dow)
>> +EXPORT(dt_from_rdn)
>> +EXPORT(dt_from_struct_tm)
>> +EXPORT(dt_from_yd)
>> +EXPORT(dt_from_ymd)
>> +EXPORT(dt_from_yqd)
>> +EXPORT(dt_from_ywd)
>> +EXPORT(dt_parse_iso_date)
>> +EXPORT(dt_parse_iso_time_basic)
>> +EXPORT(dt_parse_iso_time_extended)
>> +EXPORT(dt_parse_iso_time)
>> +EXPORT(dt_parse_iso_zone_basic)
>> +EXPORT(dt_parse_iso_zone_extended)
>> +EXPORT(dt_parse_iso_zone_lenient)
>> +EXPORT(dt_parse_iso_zone)
>> +EXPORT(dt_rdn)
>> +EXPORT(dt_to_struct_tm)
>> +EXPORT(dt_to_yd)
>> +EXPORT(dt_to_ymd)
>> +EXPORT(dt_to_yqd)
>> +EXPORT(dt_to_ywd)
> 
> 1. In scope of https://github.com/tarantool/tarantool/pull/6265 the exported
> symbols are heavily reworked. In particular, you should not export library
> symbols as is, you need to wrap them into tt_cdt_* functions and export them
> instead. Otherwise the users' modules, which link with their own version of
> c-dt, will have conflicts.

OMG, Shit, that was unfortunate collision.

Leonid is that the recommended way now forward? What should we do within 
patchset which is on the flight at the moment?

> 
>>   EXPORT(error_ref)
>>   EXPORT(error_set_prev)
>>   EXPORT(error_unpack_unsafe)
>> diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
>> new file mode 100644
>> index 000000000..e77b188b7
>> --- /dev/null
>> +++ b/src/lib/core/datetime.c
>> @@ -0,0 +1,96 @@
>> +/*
>> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
>> + *
>> + * Redistribution and use in source and binary forms, with or
>> + * without modification, are permitted provided that the following
>> + * conditions are met:
>> + *
>> + * 1. Redistributions of source code must retain the above
>> + *    copyright notice, this list of conditions and the
>> + *    following disclaimer.
>> + *
>> + * 2. Redistributions in binary form must reproduce the above
>> + *    copyright notice, this list of conditions and the following
>> + *    disclaimer in the documentation and/or other materials
>> + *    provided with the distribution.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
>> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
>> + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
>> + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
>> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
>> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
>> + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
>> + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
>> + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>> + * SUCH DAMAGE.
>> + */
>> +
>> +#include <string.h>
>> +#include <time.h>
>> +
>> +#include "trivia/util.h"
>> +#include "datetime.h"
>> +
>> +static int
>> +local_dt(int64_t secs)
>> +{
>> +	return dt_from_rdn((int)(secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET);
>> +}
>> +
>> +static struct tm *
>> +datetime_to_tm(const struct datetime *date)
>> +{
>> +	static struct tm tm;
>> +
>> +	memset(&tm, 0, sizeof(tm));
> 
> 2. Why do you need this memset if you overwrite all the
> fields anyway?

Not, all fields, there are (at least) 3 important fields left untouched: 
tm_isdst, tm_gmtoff, and tm_tmzone. I have to reset them every time this 
function got called.

> 
>> +	int64_t secs = date->secs;
>> +	dt_to_struct_tm(local_dt(secs), &tm);
>> +
>> +	int seconds_of_day = date->secs % SECS_PER_DAY;
>> +	tm.tm_hour = (seconds_of_day / 3600) % 24;
>> +	tm.tm_min = (seconds_of_day / 60) % 60;
>> +	tm.tm_sec = seconds_of_day % 60;
>> +
>> +	return &tm;
>> +}
>> +
>> +void
>> +datetime_now(struct datetime * now)
> 
> 3. Please, do not use whitespace between * and the variable name.

Yup.

> 
>> +{
>> +	struct timeval tv;
>> +	gettimeofday(&tv, NULL);
>> +	now->secs = tv.tv_sec;
>> +	now->nsec = tv.tv_usec * 1000;
>> +
>> +	time_t now_seconds;
>> +	time(&now_seconds);
>> +	struct tm tm;
>> +	localtime_r(&now_seconds, &tm);
> 
> 4. Wtf? 2 system calls just to get the current time? Can you use
> just one? Why can't you use tv.tv_sec which you got above?

gettimeofday to at least to get microseconds precision for secs/nsec. 
And time() + localtime() for timezone information. We do not actually 
interested to see time from time() + localtime() - only offset information.

Please give me know if there is some obvious, simpler way to retrieve 
nanoseconds and timezone in less number of libc calls?


> 
>> +	now->offset = tm.tm_gmtoff / 60;
>> +}
>> +
>> +char *
>> +datetime_asctime(const struct datetime *date)
>> +{
>> +	struct tm *p_tm = datetime_to_tm(date);
>> +	return asctime(p_tm);
> 
> 5. You can't return a global buffer. You need to write the
> result into a local one passed in the arguments. AFAIS, there
> are functions with _r suffix which should help.

Yes, reentrant version of those functions should help, will update patch 
soon. Fortunately we already have unit test 
test/app-tap/gh-5632-6050-6259-gc-buf-reuse.test.lua which checks GC 
efficiently here. Thanks for the hint in prior email!

> 
>> +}
>> +
>> +char *
>> +datetime_ctime(const struct datetime *date)
>> +{
>> +	time_t time = date->secs;
>> +	return ctime(&time);
>> +}

The same as above.

>> +
>> +size_t
>> +datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
>> +		  uint32_t len)
>> +{
>> +	struct tm *p_tm = datetime_to_tm(date);
>> +	return strftime(buf, len, fmt, p_tm);
>> +}
>> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
>> new file mode 100644
>> index 000000000..6699b31d3
>> --- /dev/null
>> +++ b/src/lib/core/datetime.h
>> @@ -0,0 +1,94 @@
>> +#pragma once
>> +/*
>> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
>> + *
>> + * Redistribution and use in source and binary forms, with or
>> + * without modification, are permitted provided that the following
>> + * conditions are met:
>> + *
>> + * 1. Redistributions of source code must retain the above
>> + *    copyright notice, this list of conditions and the
>> + *    following disclaimer.
>> + *
>> + * 2. Redistributions in binary form must reproduce the above
>> + *    copyright notice, this list of conditions and the following
>> + *    disclaimer in the documentation and/or other materials
>> + *    provided with the distribution.
>> + *
>> + * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
>> + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
>> + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
>> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
>> + * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
>> + * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
>> + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
>> + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
>> + * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
>> + * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>> + * SUCH DAMAGE.
>> + */
>> +
>> +#include <stdint.h>
>> +#include <stdbool.h>
>> +#include <stdio.h>
>> +#include "c-dt/dt.h"
>> +
>> +#if defined(__cplusplus)
>> +extern "C"
>> +{
>> +#endif /* defined(__cplusplus) */
>> +
>> +#ifndef SECS_PER_DAY
>> +#define SECS_PER_DAY	86400
> 
> 6. You have a tab here after the name. Please, use whitespace.

Ok.

> 
>> +#define DT_EPOCH_1970_OFFSET 719163
>> +#endif
>> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
>> new file mode 100644
>> index 000000000..a562b6872
>> --- /dev/null
>> +++ b/src/lua/datetime.lua
>> @@ -0,0 +1,638 @@
>> +local ffi = require('ffi')
>> +
>> +ffi.cdef [[
>> +
>> +    /*
>> +    `c-dt` library functions handles properly both positive and negative `dt`
>> +    values, where `dt` is a number of dates since Rata Die date (0001-01-01).
>> +
>> +    For better compactness of our typical data in MessagePack stream we shift
>> +    root of our time to the Unix Epoch date (1970-01-01), thus our 0 is
>> +    actually dt = 719163.
>> +
>> +    So here is a simple formula how convert our epoch-based seconds to dt values
>> +        dt = (secs / 86400) + 719163
>> +    Where 719163 is an offset of Unix Epoch (1970-01-01) since Rata Die
>> +    (0001-01-01) in dates.
>> +
>> +    */
>> +    typedef int dt_t;
>> +
>> +    // dt_core.h
>> +    typedef enum {
>> +        DT_MON       = 1,
>> +        DT_MONDAY    = 1,
>> +        DT_TUE       = 2,
>> +        DT_TUESDAY   = 2,
>> +        DT_WED       = 3,
>> +        DT_WEDNESDAY = 3,
>> +        DT_THU       = 4,
>> +        DT_THURSDAY  = 4,
>> +        DT_FRI       = 5,
>> +        DT_FRIDAY    = 5,
>> +        DT_SAT       = 6,
>> +        DT_SATURDAY  = 6,
>> +        DT_SUN       = 7,
>> +        DT_SUNDAY    = 7,
>> +    } dt_dow_t;
>> +
>> +    dt_t     dt_from_rdn     (int n);
>> +    dt_t     dt_from_yd      (int y, int d);
>> +    dt_t     dt_from_ymd     (int y, int m, int d);
>> +    dt_t     dt_from_yqd     (int y, int q, int d);
>> +    dt_t     dt_from_ywd     (int y, int w, int d);
>> +
>> +    void     dt_to_yd        (dt_t dt, int *y, int *d);
>> +    void     dt_to_ymd       (dt_t dt, int *y, int *m, int *d);
>> +    void     dt_to_yqd       (dt_t dt, int *y, int *q, int *d);
>> +    void     dt_to_ywd       (dt_t dt, int *y, int *w, int *d);
>> +
>> +    int      dt_rdn          (dt_t dt);
>> +    dt_dow_t dt_dow          (dt_t dt);
> 
> 7. dt_dow is not used in this file. The enum dt_dow_t either.
> The same for dt_from_rdn, dt_from_yd, dt_from_yqd, dt_from_ywd,
> dt_to_yd, dt_to_ymd, dt_to_yqd, dt_to_ywd. Please, check all the
> functions.

Yes, will clean it up.

> 
> I will return later with more comments. Need to go now.
> 

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-08 16:35         ` Safin Timur via Tarantool-patches
@ 2021-08-10 12:20           ` Vladislav Shpilevoy via Tarantool-patches
  2021-08-10 12:21             ` Igor Munkin via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-10 12:20 UTC (permalink / raw)
  To: Safin Timur, olegrok; +Cc: tarantool-patches

Hi! Thanks for the fixes!

On 08.08.2021 19:35, Safin Timur wrote:
> Much respect that you've found some time for review, even while being on vacation! Thanks!
> 
> On 08.08.2021 14:26, Vladislav Shpilevoy wrote:
>>>>> +local datetime_index_handlers = {
>>>>> +    unixtime = function(self)
>>>>> +        return self.secs
>>>>> +    end,
>>>>> +
>>>>> +    timestamp = function(self)
>>>>> +        return tonumber(self.secs) + self.nsec / 1e9
>>>>
>>>> 11. If you are saying the Lua number value range is enough,
>>>> why do you store them as 64bit integers in 'self' and
>>>> convert to a number only for the external API?
>>>
>>> May be I misunderstood your sentence, but let me elaborate here.
>>> I need seconds and nanoseconds kept separately for their efficient and precise handling, and for passing to c-dt.
>>>
>>> If we would keep 32-bit integers in seconds then we would be able to handle only dates upto 2038 year, thus we need 64-bit seconds for extended range.
>>>
>>> OTOH, not whole 64-bit range is practical, and required for expected in real-life datetime range values. It's not a problem that Lua number and int64_t have very different range for precise integer values. Lua number could handle integer values upto 9e15, which is corresponding to ...
>>
>> I know all that. The question is why do you keep storing cdata 64 bits numbers
>> inside of the datetime objects, if you convert them all to Lua numbers before
>> return? You could store self.secs as just number. And the other fields too. Lua
>> number is double, it does not loose precision for integers < 2^53, which should
>> be enough for the ranges you want to support. Correct?
> 
> I keep using 64-bit because the primary code operating with fields is on C-side, and we need Lua number only on rare cases when user asked for composed attribute date.timestamp. Whenever we deal with seconds within arthmetic operations or transfer to c-dt, we need integer C type, larger than 32-bit. It's good fit for int64_t.

But it is slower. Notably slower last time I benched, when I also
thought integers should be faster than doubles. But cdata 64 bit integers
are slower than plain Lua numbers. Perhaps because they involve too much
work with metatables for everything. Besides, doubles are larger than 32
bits - they are 53 bits until they start loosing int precision. And it is
just enough for you, isn't it?

>>>> 19. It can ignore some fields (here it ignores 'year'):
>>>>
>>>> tarantool> d1 = datetime({secs = 100, year = 1})
>>>> ---
>>>> ...
>>>>
>>>> tarantool> d2 = datetime({secs = 100, year = 2})
>>>> ---
>>>> ...
>>>>
>>>> tarantool> d1 == d2
>>>> ---
>>>> - true
>>>> ...
>>>>
>>>
>>> Yes, and that's by design. .secs defines 'easy_way' of initializing object via secs/nsec/offset, while .year selects 'ymd' mode. And `easy_way` has precedence. It looks reasonable for me, please, suggest your idea how to handle it even better without much redesign?
>>
>> You could for instance, by looking at a first key you see in the dictionary,
>> decide how will you parse the entire dict. If you see first 'secs', you parse
>> only secs, nsecs and offset. Everything else raises an error. If you see
>> first 'year', you raise an error for the other non-related keys.
>>
>> Ignorance of some keys does not look acceptable.
>>
>> And by the way, does it mean I can't specify a date using year, month, day,
>> hours, and secs? Why isn't it valid?
> 
> Here is the tricky part, agreed that this may be confusing. The "easy mode" operates on secs, nsec, offset, the human-readable mode operates on year, month, day, hour, minute, **second**, and tz.
> 
> There is second in human-readable mode, it's rather named differently (i.e. complete word, non-plural).
> 
> The more important question though - how to implement reasonably diagnosing of such mix-in of raw attributes, with human-radable attributes?
> 
> - I tend to believe that we should **not accept them both** in the single function, but rather accept .secs/.nsec/.offset via different entries, like datetime_new_raw vs datetime_new
> 
> ```
> date = require('datetime')
> T = date.new{year = 1970, month = 1, day = 1, minute = 0, seconds = 15, tz = 180}
> T2 = date.new_raw{secs = 0, nsec = 0, offset = 180} -- will be the same as new(0, 0, 180)
> ```
> 
> In this case we physically prevent mixing of different modes with different allowed attributes in the builder object.
> 
> What do you think?

I agree that these should not be allowed to mix. But the names above
look strange. They both seem raw. Non-raw is a string. Intuitively.

	datetime.new_date()
	datetime.new_stamp()

?

Or still you could have one new() and inside of it allow to set all the
fields. Like I asked above and like you also showed in your example:

	T = date.new{year = 1970, month = 1, day = 1, minute = 0, seconds = 15, tz = 180}

Because I still don't understand, why can't I specify an exact date with
year, day, month, and seconds inside of the day.

What if I want the precision does to nsecs? That is, the max
precision: year, month, day, nsecs.

>>>> 20. Serialization does not seem to be working in the console
>>>> despite you tried to implement it above:
>>>>
>>>> tarantool> d = datetime({secs = 100})
>>>> ---
>>>> ...
>>>>
>>>> tarantool> d
>>>> ---
>>>> - 'cdata<struct datetime_t>: 0x0103ad9c50'
>>>> ...
>>>
>>> Yup, redefining of __serialize is not enough for cdata object by whatever reason (I didn't look deep). But after the next patch(es) it's start to stringize properly.
>>
>> But don't you have __serialize implementation in this commit already?
>> (I don't remember and didn't see the latest code yet.)
> 
> Yes, I did have __serialize there, but that was working strangely and only started to behave properly (well, more or less) once I've added __tostring in the following commit. I'm not very well informed about expected console behaviour (regarding those 2 meta-methods, at least), so took that for granted. Probably, there is some bug, may be it's expected behaviour for cdata based data structures?

Does not matter much. I am not against serialization done in a separate
commit. I only don't like when it is added half-way in the second commit
and starts working at some point later.

I propose to make it atomic. Either the serialization is added here and
starts working here and is tested here, or all is done in a separate
commit.

>>>> 22. Did you really test this function?
>>>>
>>>> tarantool> dt = require('datetime')
>>>> ---
>>>> ...
>>>>
>>>> tarantool> d1 = dt()
>>>> ---
>>>> ...
>>>>
>>>> tarantool> d2 - 1
>>>> ---
>>>> - error: 'builtin/datetime.lua:487: bad argument #1 to ''typeof'' (C type expected,
>>>>       got number)'
>>>> ...
>>>
>>> All kinds of arguments checking implemented in the separate patch about interval arithmetic operations, please find a moment to look into them.
>>
>> Please, do not introduce broken functions. Make the correct ones right away and
>> add tests. The new code does not need to introduce full functionality if it
>> can be done in an atomic way, but it must not be visibly broken at least. Here
>> it is.
> 
> Ok, here is how it's looking in patchset: in 1 commit we introduce arithmetic support, but not handle well all corner cases (especially there is no support for proper year or month arithmetics). Special types for year, and months date intervals have been added in the next commit, where we handle all corner cases with bogus data.
> 
> We could:
> - either squash interval support to the original module;
> - or remove arithmetic altogether from original patch creating the module datetime.lua.
> 
> 
> What is the recommended way?

I would leave the intervals in their own commit, in which I would also
add the arithmetics. Working correctly right away.

>>>>> +
>>>>> +local function datetime_from(o)
>>>>> +    if o == nil or type(o) == 'table' then
>>>>> +        return datetime_new(o)
>>>>> +    elseif type(o) == 'string' then
>>>>> +        return parse(o)
>>>>> +    end
>>>>> +end
>>>>> +
>>>>> +local function local_now()
>>>>> +    local p_tv = ffi.new('struct timeval [1]')
>>>>> +    local rc = builtin.gettimeofday(p_tv, nil)
>>>>
>>>> 24. Why can't you use fiber time? It should be enough in
>>>> nearly all usecases and is much faster. For real-real time
>>>> talking directly to the kernel we have 'clock' Lua module
>>>> already. You could check if results of clock functions can
>>>> be passed into your module to create 'real' timestamps not
>>>> depending on the eventloop.
>>>
>>> Fiber time, which actually return cached ev_rt_now, looks imprecise and depends on event loop circumstances, though eventually it calls the same clock_gettime(). So fiber-time is looking like a cached timestamp value from [hopefully, non-distant] past.
>>>
>>> I'd rather prefer to have more realtime values, and clock_realtime*/clock_monotonic* look relevant, they call clock_gettime() for their purposes, which in turn calls the same gettimeofday(). So in this case we call same api, but with 2 extra call layers above.
>>
>> By using gettimeofday you drag a lot of cdef garbage including the function
>> itself and what it returns. Please, don't. Anyway, the standard says struct
>> timeval can have more members than the usually used seconds and microseconds:
>>
>>     https://pubs.opengroup.org/onlinepubs/7908799/xsh/systime.h.html
>>
>>     The <sys/time.h> header defines the timeval structure that includes
>>     **at least** the following members
>>
>>> IMHO, not worth extra complexity if we could just simply call gettimeofday().
>>
>> Now you added a lot of extra complexity exactly because of this function.
> 
> Once again, this code is in C now, there is no any non-portable timeval definition of ffi to gettimeofday in Lua - everything is in C, which extra complexity you are talking about? I do not understand.

Sorry, I didn't see the latest version at the time I wrote this
comment.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-10 12:20           ` Vladislav Shpilevoy via Tarantool-patches
@ 2021-08-10 12:21             ` Igor Munkin via Tarantool-patches
  2021-08-12 20:47               ` Safin Timur via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Igor Munkin via Tarantool-patches @ 2021-08-10 12:21 UTC (permalink / raw)
  To: Vladislav Shpilevoy; +Cc: tarantool-patches

Vlad,

On 10.08.21, Vladislav Shpilevoy wrote:
> Hi! Thanks for the fixes!
> 
> On 08.08.2021 19:35, Safin Timur wrote:
> > Much respect that you've found some time for review, even while being on vacation! Thanks!
> > 
> > On 08.08.2021 14:26, Vladislav Shpilevoy wrote:
> >>>>> +local datetime_index_handlers = {
> >>>>> +    unixtime = function(self)
> >>>>> +        return self.secs
> >>>>> +    end,
> >>>>> +
> >>>>> +    timestamp = function(self)
> >>>>> +        return tonumber(self.secs) + self.nsec / 1e9
> >>>>
> >>>> 11. If you are saying the Lua number value range is enough,
> >>>> why do you store them as 64bit integers in 'self' and
> >>>> convert to a number only for the external API?
> >>>
> >>> May be I misunderstood your sentence, but let me elaborate here.
> >>> I need seconds and nanoseconds kept separately for their efficient and precise handling, and for passing to c-dt.
> >>>
> >>> If we would keep 32-bit integers in seconds then we would be able to handle only dates upto 2038 year, thus we need 64-bit seconds for extended range.
> >>>
> >>> OTOH, not whole 64-bit range is practical, and required for expected in real-life datetime range values. It's not a problem that Lua number and int64_t have very different range for precise integer values. Lua number could handle integer values upto 9e15, which is corresponding to ...
> >>
> >> I know all that. The question is why do you keep storing cdata 64 bits numbers
> >> inside of the datetime objects, if you convert them all to Lua numbers before
> >> return? You could store self.secs as just number. And the other fields too. Lua
> >> number is double, it does not loose precision for integers < 2^53, which should
> >> be enough for the ranges you want to support. Correct?
> > 
> > I keep using 64-bit because the primary code operating with fields is on C-side, and we need Lua number only on rare cases when user asked for composed attribute date.timestamp. Whenever we deal with seconds within arthmetic operations or transfer to c-dt, we need integer C type, larger than 32-bit. It's good fit for int64_t.
> 
> But it is slower. Notably slower last time I benched, when I also
> thought integers should be faster than doubles. But cdata 64 bit integers
> are slower than plain Lua numbers. Perhaps because they involve too much
> work with metatables for everything. Besides, doubles are larger than 32
> bits - they are 53 bits until they start loosing int precision. And it is
> just enough for you, isn't it?

Sorry for breaking into the party, but reasoning is much simpler: Lua
numbers are just double values stored on the guest stack (or other
TValue storage); cdata 64-bit integers are GCcdata objects, so like all
others GC objects it has its lifetime, has to be allocated and freed and
traversed by GC. You can consider them as GCstr except they are not
interned. Hence, if the precision is fine (and it is AFAICS), then there
is not need to use GCcdata instead of Lua native numbers. In other
words, I totally agree with you.

> 

<snipped>


-- 
Best regards,
IM

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-10 12:21             ` Igor Munkin via Tarantool-patches
@ 2021-08-12 20:47               ` Safin Timur via Tarantool-patches
  2021-08-15 20:52                 ` Safin Timur via Tarantool-patches
  0 siblings, 1 reply; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-12 20:47 UTC (permalink / raw)
  To: Igor Munkin, Vladislav Shpilevoy; +Cc: tarantool-patches



On 10.08.2021 15:21, Igor Munkin wrote:
> Vlad,
> 
> On 10.08.21, Vladislav Shpilevoy wrote:
>> Hi! Thanks for the fixes!
>>
>> On 08.08.2021 19:35, Safin Timur wrote:
>>> Much respect that you've found some time for review, even while being on vacation! Thanks!
>>>
>>> On 08.08.2021 14:26, Vladislav Shpilevoy wrote:
>>>>>>> +local datetime_index_handlers = {
>>>>>>> +    unixtime = function(self)
>>>>>>> +        return self.secs
>>>>>>> +    end,
>>>>>>> +
>>>>>>> +    timestamp = function(self)
>>>>>>> +        return tonumber(self.secs) + self.nsec / 1e9
>>>>>>
>>>>>> 11. If you are saying the Lua number value range is enough,
>>>>>> why do you store them as 64bit integers in 'self' and
>>>>>> convert to a number only for the external API?
>>>>>
>>>>> May be I misunderstood your sentence, but let me elaborate here.
>>>>> I need seconds and nanoseconds kept separately for their efficient and precise handling, and for passing to c-dt.
>>>>>
>>>>> If we would keep 32-bit integers in seconds then we would be able to handle only dates upto 2038 year, thus we need 64-bit seconds for extended range.
>>>>>
>>>>> OTOH, not whole 64-bit range is practical, and required for expected in real-life datetime range values. It's not a problem that Lua number and int64_t have very different range for precise integer values. Lua number could handle integer values upto 9e15, which is corresponding to ...
>>>>
>>>> I know all that. The question is why do you keep storing cdata 64 bits numbers
>>>> inside of the datetime objects, if you convert them all to Lua numbers before
>>>> return? You could store self.secs as just number. And the other fields too. Lua
>>>> number is double, it does not loose precision for integers < 2^53, which should
>>>> be enough for the ranges you want to support. Correct?
>>>
>>> I keep using 64-bit because the primary code operating with fields is on C-side, and we need Lua number only on rare cases when user asked for composed attribute date.timestamp. Whenever we deal with seconds within arthmetic operations or transfer to c-dt, we need integer C type, larger than 32-bit. It's good fit for int64_t.
>>
>> But it is slower. Notably slower last time I benched, when I also
>> thought integers should be faster than doubles. But cdata 64 bit integers
>> are slower than plain Lua numbers. Perhaps because they involve too much
>> work with metatables for everything. Besides, doubles are larger than 32
>> bits - they are 53 bits until they start loosing int precision. And it is
>> just enough for you, isn't it?
> 
> Sorry for breaking into the party, but reasoning is much simpler: Lua
> numbers are just double values stored on the guest stack (or other
> TValue storage); cdata 64-bit integers are GCcdata objects, so like all
> others GC objects it has its lifetime, has to be allocated and freed and
> traversed by GC. You can consider them as GCstr except they are not
> interned. Hence, if the precision is fine (and it is AFAICS), then there
> is not need to use GCcdata instead of Lua native numbers. In other
> words, I totally agree with you.
> 
>>
> 
> <snipped>
> 
> 


Either I do something totally wrong, or may be I'm absolutely clueless. 
Oh rather both...

Here is experiment I've proceed 
https://gist.github.com/tsafin/f7f21aad53f23801839b3b278cfac380

I try to compare speed of accessing datetime.secs field:
- when it is current int64_t redirected via ffi to the `struct datetime`;
- when it is wrapped as (calculated) attribute of type double which we 
access via FFI datetime_secs() function accessor;
- or when it's declared as `double` in the similar `struct datetime_double`.

No surpise that calculated attribute is 3x orders of magnitude slower 
than direct ffi access to either int64_t or double field.

OTOH, differences for timings to access to int64_t (which should be 
boxed) and double (which should be unboxed) are negligible, and 
essentially the same:

```
✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
ctype<struct datetime>
date.secs       0.00035929679870605
ctype<struct datetime>
date.secsf      0.49544525146484
ctype<struct datetime_double>
date_double.secs        0.00042939186096191
✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
ctype<struct datetime>
date.secs       0.00034856796264648
ctype<struct datetime>
date.secsf      0.40926361083984
ctype<struct datetime_double>
date_double.secs        0.00043344497680664
✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
ctype<struct datetime>
date.secs       0.00034213066101074
ctype<struct datetime>
date.secsf      0.46818256378174
ctype<struct datetime_double>
date_double.secs        0.00037813186645508
✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
ctype<struct datetime>
date.secs       0.00051259994506836
ctype<struct datetime>
date.secsf      0.6695671081543
ctype<struct datetime_double>
date_double.secs        0.00048208236694336
```

What did I do wrong?

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime
  2021-08-12 20:47               ` Safin Timur via Tarantool-patches
@ 2021-08-15 20:52                 ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 38+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-15 20:52 UTC (permalink / raw)
  To: Igor Munkin, Vladislav Shpilevoy; +Cc: tarantool-patches

There are updates with jot.on() vs jit.off() numbers, and further 
investigations for C benchmarks of double vs int64_t, please see below...

On 12.08.2021 23:47, Safin Timur via Tarantool-patches wrote:
> 
> 
> On 10.08.2021 15:21, Igor Munkin wrote:
>> Vlad,
>>
>> On 10.08.21, Vladislav Shpilevoy wrote:
>>> Hi! Thanks for the fixes!
>>>
>>> On 08.08.2021 19:35, Safin Timur wrote:
>>>> Much respect that you've found some time for review, even while 
>>>> being on vacation! Thanks!
>>>>
>>>> On 08.08.2021 14:26, Vladislav Shpilevoy wrote:
>>>>>>>> +local datetime_index_handlers = {
>>>>>>>> +    unixtime = function(self)
>>>>>>>> +        return self.secs
>>>>>>>> +    end,
>>>>>>>> +
>>>>>>>> +    timestamp = function(self)
>>>>>>>> +        return tonumber(self.secs) + self.nsec / 1e9
>>>>>>>
>>>>>>> 11. If you are saying the Lua number value range is enough,
>>>>>>> why do you store them as 64bit integers in 'self' and
>>>>>>> convert to a number only for the external API?
>>>>>>
>>>>>> May be I misunderstood your sentence, but let me elaborate here.
>>>>>> I need seconds and nanoseconds kept separately for their efficient 
>>>>>> and precise handling, and for passing to c-dt.
>>>>>>
>>>>>> If we would keep 32-bit integers in seconds then we would be able 
>>>>>> to handle only dates upto 2038 year, thus we need 64-bit seconds 
>>>>>> for extended range.
>>>>>>
>>>>>> OTOH, not whole 64-bit range is practical, and required for 
>>>>>> expected in real-life datetime range values. It's not a problem 
>>>>>> that Lua number and int64_t have very different range for precise 
>>>>>> integer values. Lua number could handle integer values upto 9e15, 
>>>>>> which is corresponding to ...
>>>>>
>>>>> I know all that. The question is why do you keep storing cdata 64 
>>>>> bits numbers
>>>>> inside of the datetime objects, if you convert them all to Lua 
>>>>> numbers before
>>>>> return? You could store self.secs as just number. And the other 
>>>>> fields too. Lua
>>>>> number is double, it does not loose precision for integers < 2^53, 
>>>>> which should
>>>>> be enough for the ranges you want to support. Correct?
>>>>
>>>> I keep using 64-bit because the primary code operating with fields 
>>>> is on C-side, and we need Lua number only on rare cases when user 
>>>> asked for composed attribute date.timestamp. Whenever we deal with 
>>>> seconds within arthmetic operations or transfer to c-dt, we need 
>>>> integer C type, larger than 32-bit. It's good fit for int64_t.
>>>
>>> But it is slower. Notably slower last time I benched, when I also
>>> thought integers should be faster than doubles. But cdata 64 bit 
>>> integers
>>> are slower than plain Lua numbers. Perhaps because they involve too much
>>> work with metatables for everything. Besides, doubles are larger than 32
>>> bits - they are 53 bits until they start loosing int precision. And 
>>> it is
>>> just enough for you, isn't it?
>>
>> Sorry for breaking into the party, but reasoning is much simpler: Lua
>> numbers are just double values stored on the guest stack (or other
>> TValue storage); cdata 64-bit integers are GCcdata objects, so like all
>> others GC objects it has its lifetime, has to be allocated and freed and
>> traversed by GC. You can consider them as GCstr except they are not
>> interned. Hence, if the precision is fine (and it is AFAICS), then there
>> is not need to use GCcdata instead of Lua native numbers. In other
>> words, I totally agree with you.
>>
>>>
>>
>> <snipped>
>>
>>
> 
> 
> Either I do something totally wrong, or may be I'm absolutely clueless.
> Oh rather both...
> 
> Here is experiment I've proceed
> https://gist.github.com/tsafin/f7f21aad53f23801839b3b278cfac380
> 
> I try to compare speed of accessing datetime.secs field:
> - when it is current int64_t redirected via ffi to the `struct datetime`;
> - when it is wrapped as (calculated) attribute of type double which we
> access via FFI datetime_secs() function accessor;
> - or when it's declared as `double` in the similar `struct 
> datetime_double`.
> 
> No surpise that calculated attribute is 3x orders of magnitude slower
> than direct ffi access to either int64_t or double field.
> 
> OTOH, differences for timings to access to int64_t (which should be
> boxed) and double (which should be unboxed) are negligible, and
> essentially the same:
> 
> ```
> ✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
> 23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
> ctype<struct datetime>
> date.secs       0.00035929679870605
> ctype<struct datetime>
> date.secsf      0.49544525146484
> ctype<struct datetime_double>
> date_double.secs        0.00042939186096191
> ✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
> 23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
> ctype<struct datetime>
> date.secs       0.00034856796264648
> ctype<struct datetime>
> date.secsf      0.40926361083984
> ctype<struct datetime_double>
> date_double.secs        0.00043344497680664
> ✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
> 23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
> ctype<struct datetime>
> date.secs       0.00034213066101074
> ctype<struct datetime>
> date.secsf      0.46818256378174
> ctype<struct datetime_double>
> date_double.secs        0.00037813186645508
> ✔ ~/datetime/tarantoolt/build [tsafin/gh-5941-datetime-v4 ↑·4|✚ 3…4⚑ 3]
> 23:40 $ ./src/tarantool ../../bench-datetime-secs.lua
> ctype<struct datetime>
> date.secs       0.00051259994506836
> ctype<struct datetime>
> date.secsf      0.6695671081543
> ctype<struct datetime_double>
> date_double.secs        0.00048208236694336
> ```
> 
> What did I do wrong?
> 
> Thanks,
> Timur

After discussions with Igor it was decided to investiage differences in 
behavior of Lua code when jit.on() or off().

https://gist.github.com/tsafin/f7f21aad53f23801839b3b278cfac380#file-jit-on-vs-off-md

jit.off()
=========


```
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:47 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.25970935821533
date_double.secs        0.15149474143982
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:48 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.33055567741394
date_double.secs        0.18803310394287
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:48 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.29576468467712
date_double.secs        0.15472149848938
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:48 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.2386200428009
date_double.secs        0.1287829875946
```

jit.on()
========
```
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:46 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.0029795169830322
date_double.secs        0.00043296813964844
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:47 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.00053858757019043
date_double.secs        0.00038242340087891
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:47 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.00038433074951172
date_double.secs        0.00033235549926758
✔ ~/datetime/tarantoolt/build [tsafin/msgpack-format L|✚ 1…6⚑ 3]
14:47 $ ./src/tarantool ../../bench-hash-dt.lua
date.secs       0.0003509521484375
date_double.secs        0.00031447410583496
```
i.e. for jit.on() mode there is no much difference in timings of boxed 
(ffi.int64_t) or unboxed (double) types used. Igor assumes that type 
narrowing was working well in this case.

But with jit.off() we have dramatically diferent picture - double field 
is clearly faster than int64_t field. Upto 2x faster.

So it was interesting to see how it looks from the other side - if we 
compare C code with structures using double instead of int64_t.

Please see (shocking for me) results here -
https://gist.github.com/tsafin/618fbf847d258f6e7f5a75fdf9ea945b

Here is excerpt from result for SSE2 mode (which is default):

| Test Name                             | Median  | slower/faster |
|---------------------------------------|---------|-----|
| `DateTime_Assign70<uint64_t>`         | 5254 ns |  -  |
| `DateTime_Assign70<double>`           | 4948 ns |  +  |
| `DateTime_AssignCompare70<uint64_t>`  | 4948 ns |  -  |
| `DateTime_AssignCompare70<double>`    | 4784 ns |  +  |
| `DateTime_Compare20<uint64_t>`        | 4 ns    |  -  |
| `DateTime_Compare20<double>`          | 3 ns    |  +  |
| `DateTime_ToString1<uint64_t>`        | 515 ns  |  +  |
| `DateTime_ToString1<double>`          | 522 ns  |  -  |

Comparison shows that double provides similar or sometimes better 
timings, despite our prior experience with older x86 processors. (Be it 
SSE2, AVX1 or AVX2 code).

The only benchmark which has been constantly slower with doubles -
was stringization `DatetTime_ToString` which is consistently showing 
worse numbers with doubles than with int64s. But difference is very 
small and negligible in a longer run.

So, I'm finally convinced (data-driven decision!) that double might be a 
good type for using within `datetime.secs`, noth from LuaJIT and C point 
of view.


If you are curious enough, please see below the additional patch with 
benchmark code, which I've pushed to the branch...
-----------------------------------------------------------------
commit f2080cc7df2567cf2ec66d8ad341572ae76a77a4
Author: Timur Safin <tsafin@tarantool.org>
Date:   Sun Aug 15 17:19:26 2021 +0300

     datetime: perf test for datetime parser

     It was told that if field `datetime.secs` would be `double` we 
should get
     better performance in LuaJIT instead of `uint64_t` type, which is 
used at the
     moment.

     So we have created benchmark, which was comparing implementations 
of functions
     from `datetime.c` if we would use `double` or `int64_t` for 
`datetime.secs` field.

     Despite expectations, based on prior experience with floaing-point 
on x86
     processors, comparison shows that `double` provides similar or
     sometimes better timings. And picture stays consistent be it SSE2, 
AVX1 or
     AVX2 code.

diff --git a/perf/CMakeLists.txt b/perf/CMakeLists.txt
index 3651de5b4..b5d7caf81 100644
--- a/perf/CMakeLists.txt
+++ b/perf/CMakeLists.txt
@@ -12,3 +12,6 @@ include_directories(${CMAKE_SOURCE_DIR}/third_party)

  add_executable(tuple.perftest tuple.cc)
  target_link_libraries(tuple.perftest core box tuple benchmark::benchmark)
+
+add_executable(datetime.perftest datetime-parser.cc datetime-compare.cc)
+target_link_libraries(datetime.perftest cdt core benchmark::benchmark)
diff --git a/perf/datetime-common.h b/perf/datetime-common.h
new file mode 100644
index 000000000..6fd4e1e3b
--- /dev/null
+++ b/perf/datetime-common.h
@@ -0,0 +1,105 @@
+#include <assert.h>
+#include <stdint.h>
+#include <string.h>
+#include <benchmark/benchmark.h>
+
+#include "dt.h"
+#include "datetime.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+static const char sample[] = "2012-12-24T15:30Z";
+
+#define S(s)               \
+	{ s, sizeof(s) - 1 }
+
+static struct
+{
+	const char *sz;
+	size_t len;
+} tests[] = {
+    S("2012-12-24 15:30Z"),
+    S("2012-12-24 15:30z"),
+    S("2012-12-24 15:30"),
+    S("2012-12-24 16:30+01:00"),
+    S("2012-12-24 16:30+0100"),
+    S("2012-12-24 16:30+01"),
+    S("2012-12-24 14:30-01:00"),
+    S("2012-12-24 14:30-0100"),
+    S("2012-12-24 14:30-01"),
+    S("2012-12-24 15:30:00Z"),
+    S("2012-12-24 15:30:00z"),
+    S("2012-12-24 15:30:00"),
+    S("2012-12-24 16:30:00+01:00"),
+    S("2012-12-24 16:30:00+0100"),
+    S("2012-12-24 14:30:00-01:00"),
+    S("2012-12-24 14:30:00-0100"),
+    S("2012-12-24 15:30:00.123456Z"),
+    S("2012-12-24 15:30:00.123456z"),
+    S("2012-12-24 15:30:00.123456"),
+    S("2012-12-24 16:30:00.123456+01:00"),
+    S("2012-12-24 16:30:00.123456+01"),
+    S("2012-12-24 14:30:00.123456-01:00"),
+    S("2012-12-24 14:30:00.123456-01"),
+    S("2012-12-24t15:30Z"),
+    S("2012-12-24t15:30z"),
+    S("2012-12-24t15:30"),
+    S("2012-12-24t16:30+01:00"),
+    S("2012-12-24t16:30+0100"),
+    S("2012-12-24t14:30-01:00"),
+    S("2012-12-24t14:30-0100"),
+    S("2012-12-24t15:30:00Z"),
+    S("2012-12-24t15:30:00z"),
+    S("2012-12-24t15:30:00"),
+    S("2012-12-24t16:30:00+01:00"),
+    S("2012-12-24t16:30:00+0100"),
+    S("2012-12-24t14:30:00-01:00"),
+    S("2012-12-24t14:30:00-0100"),
+    S("2012-12-24t15:30:00.123456Z"),
+    S("2012-12-24t15:30:00.123456z"),
+    S("2012-12-24t16:30:00.123456+01:00"),
+    S("2012-12-24t14:30:00.123456-01:00"),
+    S("2012-12-24 16:30 +01:00"),
+    S("2012-12-24 14:30 -01:00"),
+    S("2012-12-24 15:30 UTC"),
+    S("2012-12-24 16:30 UTC+1"),
+    S("2012-12-24 16:30 UTC+01"),
+    S("2012-12-24 16:30 UTC+0100"),
+    S("2012-12-24 16:30 UTC+01:00"),
+    S("2012-12-24 14:30 UTC-1"),
+    S("2012-12-24 14:30 UTC-01"),
+    S("2012-12-24 14:30 UTC-01:00"),
+    S("2012-12-24 14:30 UTC-0100"),
+    S("2012-12-24 15:30 GMT"),
+    S("2012-12-24 16:30 GMT+1"),
+    S("2012-12-24 16:30 GMT+01"),
+    S("2012-12-24 16:30 GMT+0100"),
+    S("2012-12-24 16:30 GMT+01:00"),
+    S("2012-12-24 14:30 GMT-1"),
+    S("2012-12-24 14:30 GMT-01"),
+    S("2012-12-24 14:30 GMT-01:00"),
+    S("2012-12-24 14:30 GMT-0100"),
+    S("2012-12-24 14:30 -01:00"),
+    S("2012-12-24 16:30:00 +01:00"),
+    S("2012-12-24 14:30:00 -01:00"),
+    S("2012-12-24 16:30:00.123456 +01:00"),
+    S("2012-12-24 14:30:00.123456 -01:00"),
+    S("2012-12-24 15:30:00.123456 -00:00"),
+    S("20121224T1630+01:00"),
+    S("2012-12-24T1630+01:00"),
+    S("20121224T16:30+01"),
+    S("20121224T16:30 +01"),
+};
+#undef S
+
+#define DIM(a) (sizeof(a) / sizeof(a[0]))
+
+int
+parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
+	       int32_t *op);
+
+#ifdef __cplusplus
+}
+#endif
diff --git a/perf/datetime-compare.cc b/perf/datetime-compare.cc
new file mode 100644
index 000000000..efd0dba0e
--- /dev/null
+++ b/perf/datetime-compare.cc
@@ -0,0 +1,212 @@
+#include "dt.h"
+#include <string.h>
+#include <assert.h>
+
+#include "datetime-common.h"
+
+template <typename T>
+struct datetime_bench
+{
+	T secs;
+	uint32_t nsec;
+	uint32_t offset;
+
+static struct datetime_bench date_array[];
+};
+template<typename T>
+struct datetime_bench<T> datetime_bench<T>::date_array[DIM(tests)];
+
+/// Parse 70 datetime literals of various lengths
+template <typename T>
+static void
+Assign70()
+{
+	size_t index;
+	int64_t secs_expected;
+	int nanosecs;
+	int ofs;
+	using dt_bench = datetime_bench<T>;
+
+	for (index = 0; index < DIM(tests); index++) {
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+					&secs, &nanosecs, &ofs);
+		assert(rc == 0);
+		dt_bench::date_array[index].secs = (T)secs;
+		dt_bench::date_array[index].nsec = nanosecs;
+		dt_bench::date_array[index].offset = ofs;
+	}
+}
+
+template <typename T>
+static void
+DateTime_Assign70(benchmark::State &state)
+{
+	for (auto _ : state)
+		Assign70<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_Assign70, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_Assign70, double);
+
+#define COMPARE_RESULT_BENCH(a, b) (a < b ? -1 : a > b)
+
+template <typename T>
+int datetime_compare(const struct datetime_bench<T> *lhs,
+		     const struct datetime_bench<T> *rhs)
+{
+	int result = COMPARE_RESULT_BENCH(lhs->secs, rhs->secs);
+	if (result != 0)
+		return result;
+
+	return COMPARE_RESULT_BENCH(lhs->nsec, rhs->nsec);
+}
+
+template <typename T>
+static void
+AssignCompare70()
+{
+	size_t index;
+	int nanosecs;
+	int ofs;
+	using dt_bench = datetime_bench<T>;
+
+	size_t arrays_sz = DIM(tests);
+	for (index = 0; index < arrays_sz; index++) {
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+					&secs, &nanosecs, &ofs);
+		assert(rc == 0);
+		dt_bench::date_array[index].secs = (T)secs;
+		dt_bench::date_array[index].nsec = nanosecs;
+		dt_bench::date_array[index].offset = ofs;
+	}
+
+	for (index = 0; index < (arrays_sz - 1); index++) {
+		volatile int rc = datetime_compare<T>(&dt_bench::date_array[index],
+					     &dt_bench::date_array[index + 1]);
+		assert(rc == 0 || rc == -1 || rc == 1);
+	}
+}
+
+template <typename T>
+static void
+DateTime_AssignCompare70(benchmark::State &state)
+{
+	for (auto _ : state)
+		AssignCompare70<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_AssignCompare70, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_AssignCompare70, double);
+
+template <typename T>
+static void
+Compare20()
+{
+	size_t index;
+	int nanosecs;
+	int ofs;
+	using dt_bench = datetime_bench<T>;
+
+	for (size_t i = 0; i < 10; i++) {
+		volatile int rc = datetime_compare<T>(&dt_bench::date_array[i],
+					     &dt_bench::date_array[32 + i]);
+		assert(rc == 0 || rc == -1 || rc == 1);
+	}
+}
+
+template <typename T>
+static void
+DateTime_Compare20(benchmark::State &state)
+{
+	for (auto _ : state)
+		Compare20<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_Compare20, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_Compare20, double);
+
+
+#define SECS_EPOCH_1970_OFFSET ((int64_t)DT_EPOCH_1970_OFFSET * 
SECS_PER_DAY)
+
+template<typename T>
+int
+datetime_to_string(const struct datetime_bench<T> *date, char *buf, 
uint32_t len)
+{
+#define ADVANCE(sz)		\
+	if (buf != NULL) { 	\
+		buf += sz; 	\
+		len -= sz; 	\
+	}			\
+	ret += sz;
+
+	int offset = date->offset;
+	/* for negative offsets around Epoch date we could get
+	 * negative secs value, which should be attributed to
+	 * 1969-12-31, not 1970-01-01, thus we first shift
+	 * epoch to Rata Die then divide by seconds per day,
+	 * not in reverse
+	 */
+	int64_t secs = (int64_t)date->secs + offset * 60 + SECS_EPOCH_1970_OFFSET;
+	assert((secs / SECS_PER_DAY) <= INT_MAX);
+	dt_t dt = dt_from_rdn(secs / SECS_PER_DAY);
+
+	int year, month, day, sec, ns, sign;
+	dt_to_ymd(dt, &year, &month, &day);
+
+	int hour = (secs / 3600) % 24,
+	    minute = (secs / 60) % 60;
+	sec = secs % 60;
+	ns = date->nsec;
+
+	int ret = 0;
+	uint32_t sz = snprintf(buf, len, "%04d-%02d-%02dT%02d:%02d",
+			       year, month, day, hour, minute);
+	ADVANCE(sz);
+	if (sec || ns) {
+		sz = snprintf(buf, len, ":%02d", sec);
+		ADVANCE(sz);
+		if (ns) {
+			if ((ns % 1000000) == 0)
+				sz = snprintf(buf, len, ".%03d", ns / 1000000);
+			else if ((ns % 1000) == 0)
+				sz = snprintf(buf, len, ".%06d", ns / 1000);
+			else
+				sz = snprintf(buf, len, ".%09d", ns);
+			ADVANCE(sz);
+		}
+	}
+	if (offset == 0) {
+		sz = snprintf(buf, len, "Z");
+		ADVANCE(sz);
+	}
+	else {
+		if (offset < 0)
+			sign = '-', offset = -offset;
+		else
+			sign = '+';
+
+		sz = snprintf(buf, len, "%c%02d:%02d", sign, offset / 60, offset % 60);
+		ADVANCE(sz);
+	}
+	return ret;
+}
+#undef ADVANCE
+
+template <typename T>
+static void
+ToString1()
+{
+	char buf[48];
+	struct datetime_bench<T> dateval = datetime_bench<T>::date_array[13];
+
+	volatile auto len = datetime_to_string<T>(&dateval, buf, sizeof buf);
+}
+
+template <typename T>
+static void
+DateTime_ToString1(benchmark::State &state)
+{
+	for (auto _ : state)
+		ToString1<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_ToString1, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_ToString1, double);
diff --git a/perf/datetime-parser.cc b/perf/datetime-parser.cc
new file mode 100644
index 000000000..61557fe8f
--- /dev/null
+++ b/perf/datetime-parser.cc
@@ -0,0 +1,105 @@
+#include "dt.h"
+#include <string.h>
+#include <assert.h>
+
+#include "datetime-common.h"
+
+/* p5-time-moment/src/moment_parse.c: parse_string_lenient() */
+int
+parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
+	       int32_t *op)
+{
+	size_t n;
+	dt_t dt;
+	char c;
+	int sod = 0, nanosecond = 0, offset = 0;
+
+	n = dt_parse_iso_date(str, len, &dt);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	c = str[n++];
+	if (!(c == 'T' || c == 't' || c == ' '))
+		return 1;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_time(str, len, &sod, &nanosecond);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	if (str[n] == ' ')
+	n++;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_zone_lenient(str, len, &offset);
+	if (!n || n != len)
+		return 1;
+
+exit:
+	*sp = ((int64_t)dt_rdn(dt) - 719163) * 86400 + sod - offset * 60;
+	*np = nanosecond;
+	*op = offset;
+
+	return 0;
+}
+
+/// Parse 70 datetime literals of various lengths
+static void
+ParseTimeStamps()
+{
+	size_t index;
+	int64_t secs_expected;
+	int nanosecs;
+	int ofs;
+	parse_datetime(sample, sizeof(sample) - 1, &secs_expected,
+		       &nanosecs, &ofs);
+
+	for (index = 0; index < DIM(tests); index++)
+	{
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+					&secs, &nanosecs, &ofs);
+		assert(rc == 0);
+		assert(secs == secs_expected);
+	}
+}
+
+static void
+CDT_Parse70(benchmark::State &state)
+{
+	for (auto _ : state)
+		ParseTimeStamps();
+}
+BENCHMARK(CDT_Parse70);
+
+/// Parse single datetime literal of longest length
+static void
+Parse1()
+{
+	const char civil_string[] = "2015-02-18T10:50:31.521345123+10:00";
+	int64_t secs;
+	int nanosecs;
+	int ofs;
+	int rc = parse_datetime(civil_string, sizeof(civil_string) - 1,
+				&secs, &nanosecs, &ofs);
+	assert(rc == 0);
+	assert(nanosecs == 521345123);
+}
+
+static void
+CDT_Parse1(benchmark::State &state)
+{
+	for (auto _ : state)
+		Parse1();
+}
+BENCHMARK(CDT_Parse1);
+
+BENCHMARK_MAIN();
-----------------------------------------------------------------

Running to switch to doubles in all references to `struct datetime` now...

Thanks,
Timur

^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2021-08-15 20:53 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-02  0:40 [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches
2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 1/9] build: add Christian Hansen c-dt to the build Timur Safin via Tarantool-patches
2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
2021-08-05  8:55     ` Safin Timur via Tarantool-patches
2021-08-08 14:34       ` Vladislav Shpilevoy via Tarantool-patches
2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 2/9] lua: built-in module datetime Timur Safin via Tarantool-patches
2021-08-04 23:58   ` Vladislav Shpilevoy via Tarantool-patches
2021-08-06  0:23     ` Safin Timur via Tarantool-patches
2021-08-06  1:30       ` Oleg Babin via Tarantool-patches
2021-08-06 13:00         ` Safin Timur via Tarantool-patches
2021-08-06 17:24           ` Safin Timur via Tarantool-patches
2021-08-08 11:26       ` Vladislav Shpilevoy via Tarantool-patches
2021-08-08 16:35         ` Safin Timur via Tarantool-patches
2021-08-10 12:20           ` Vladislav Shpilevoy via Tarantool-patches
2021-08-10 12:21             ` Igor Munkin via Tarantool-patches
2021-08-12 20:47               ` Safin Timur via Tarantool-patches
2021-08-15 20:52                 ` Safin Timur via Tarantool-patches
2021-08-06  0:26   ` Safin Timur via Tarantool-patches
2021-08-08 14:34   ` Vladislav Shpilevoy via Tarantool-patches
2021-08-08 16:47     ` Safin Timur via Tarantool-patches
2021-08-02  0:40 ` [Tarantool-patches] [PATCH v3 3/9] lua, datetime: datetime tests Timur Safin via Tarantool-patches
2021-08-06  0:25   ` Safin Timur via Tarantool-patches
2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 4/9] lua, datetime: display datetime Timur Safin via Tarantool-patches
2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 5/9] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
2021-08-03 13:38   ` Timur Safin via Tarantool-patches
2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
2021-08-03 12:02   ` Serge Petrenko via Tarantool-patches
2021-08-03 12:59     ` Timur Safin via Tarantool-patches
2021-08-04 10:12     ` Timur Safin via Tarantool-patches
2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 7/9] lua, datetime: time intervals support Timur Safin via Tarantool-patches
2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 8/9] datetime: changelog for datetime module Timur Safin via Tarantool-patches
2021-08-02  0:41 ` [Tarantool-patches] [PATCH v3 9/9] lua, box, datetime: rename struct datetime_t Timur Safin via Tarantool-patches
2021-08-06  0:27   ` Safin Timur via Tarantool-patches
2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 1/2] datetime: update tests for macosx Timur Safin via Tarantool-patches
2021-08-06  0:28   ` Safin Timur via Tarantool-patches
2021-08-03 21:23 ` [Tarantool-patches] [PATCH v3 2/2] lua, datetime: introduce ctime, strftime wrappers Timur Safin via Tarantool-patches
2021-08-06  0:30   ` Safin Timur via Tarantool-patches
2021-08-03 21:26 ` [Tarantool-patches] [PATCH v3 0/9] Initial datetime support Timur Safin via Tarantool-patches

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox