Tarantool development patches archive
 help / color / mirror / Atom feed
* [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation
@ 2021-08-19  2:56 Timur Safin via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime Timur Safin via Tarantool-patches
                   ` (4 more replies)
  0 siblings, 5 replies; 20+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-19  2:56 UTC (permalink / raw)
  To: vdavydov, sergepetrenko, tarantool-patches; +Cc: v.shpilevoy

* Version #6 changes:

Massive changes and reshuffles to the patches in patchset, now we 
consolidated them to 3 major parts:

  - creation of a Lua module;
  - serialization to MessagePack and other formats;
  - Box storage support;

As a result of yesterday discussion we significantly simplified
datetime.lua API. We have dropped interval support, and made 
implementation as immutable, as possible. Now object have these 
major methods:

  - add{}
  - sub{}
  - totable()
  - set{}

Interface for constructors kept the same, but performance improved
due to removed loops over keys in hash.

We got rid of ctime()/asctime() methods, as they are useless. strftime() 
is good enough for majority of similar cases.

In addition to these massive datetime.lua changes we have reshuffled code 
for indices support - we have moved part introducing new field type from 
serialization patch, to the more appropriate box indices patch.

datetime_to_string() has been reimplemented using SNPRINT facilities.

As a bonus part - mp_utils.h has been introduced to simiplify work with 
integer values when they need to be encoded/decoded to/from MesagePack. 
This API works regardless of a values signedness.

| I'm planning to resend updated datetime.lua patch as it's not yet 
| feature-complete - there is no working set{} method
|
| Also we need to extend tests with a new API methods usage. Stay tuned!


* Version #5 changes:

  - After performance evaluations it was decided to switch data type used
    by `datetime.secs` from `int64_t` (which is boxed in LuaJIT) to
    `double` (which is unboxed number).
  - please see perf/datetime*.cc perf test as a benchmark we used for
    comparison of 2 veriants of C implementation. Results are here - 
    https://gist.github.com/tsafin/618fbf847d258f6e7f5a75fdf9ea945b

* Version #4 changes:

A lot.

  - Got rid of closures usage here and there. Benchmarks revealed there is 
    20x speed difference of hash with closure handlers to the case when we 
    check keys using sequence of simple ifs.
    And despite the fact that it looks ugler, we have updated all places which
    used to use closures - so much performance impact does not worth it;

  - c-dt library now is accompanied with corresponding unit-tests;

  - due to a new (old) way of exporting public symbols, which is now used by 
    Tarantool build, we have modified c-dt sources to use approach similar 
    to that used by xxhash (or ICU) - "namespaces" for public symbols.
    If there is DT_NAMESPACE macro (e.g. DT_NAMESPACE=tnt_) defined at the 
    moment of compilation we rename all relevant public symbols to 
    use that "namespace" as prefix
 
	#define dt_from_rdn DT_NAME(DT_NAMESPACE, dt_from_rdn)

    C sources continue to use original name (now as a macro), but in Lua ffi
    we have to use renamed symbol - `tnt_dt_from_rdn`.

  - We have separated code which implement MessagePack related funcionality
    from generic (i.e. stringization) support for datetime types.
    Stringization is in `src/core/datetime.c`, but MessagePack support is in 
    `src/core/mp_datetime.c`.

  - `ctime` and `asctime` now use reentrant versions (i.e. `ctime_r` instead of 
    `ctime`, and `asctime_r` instead of `asctime`). This allows us to avoid 
    nasty problem with GC where it was possible to corrupt returned 
    static buffer regardless the fact that it was immediately passed
    to `ffi.string`. 
    `test/app-tap/gh-5632-6050-6259-gc-buf-reuse.test.lua` updated to check our 
    new api `datetime.ctime`, and `datetime.asctime`. 

  - Introduced MessagePack unit-test which check C implementation (mp 
    encode/decode, mp_snprint, mp_fprint) 

  - Some special efforts have been taken to make sure we deal correctly 
    with datetime stamps which were before Unix Epoch (1970-01-01), i.e. 
    for negative stamps or timezones against epoch.

  - And last, but not the least: in addition to usual getters like 
    `timestamp`, `nanoseconds`, `epoch`, we have introduced function 
    modifiers `to_utc()` or `to_tz(offset)` which allow to manipulate with 
    datetime timezone `.offset` field. Actually it does not do much now 
    (only reattributing object with different offset, but not changed 
    actual timestamp, which is UTC normalized). But different timestamp 
    modifies textual representation, which simplify debugging.

* Version #3 changes:

  - renamed `struct datetime_t` to `struct datetime`, and `struct 
    datetime_interval_t` to `struct datetime_interval`;
  - significantly reworked arguments checks in module api entries;
  - fixed datetime comparisons;
  - changed hints calculation to take into account fractional part;
  - provided more comments here and there;

NB! There are MacOSX problems due to GLIBC specific code used (as Vlad
    has already pointed out) - so additional patch, making it more 
    cross-compatible coming here soon...

* Version #2 changes:

  - fixed problem with overloaded '-' and '+' operations for datetime 
    arguments;
  - fixed messagepack serialization problems;
  - heavily documented MessagePack serialization schema in the code;
  - introduced working implementation of datetime hints for storage engines;
  - made interval related names be more consistent, renamed durations and period 
    to intervals, i.e. t_datetime_duration to datetime_interval_t, 
    duration_* to interval_*, period to interval;
  - properly implemented all reasonable cases of datetime+interval 
    arithmetic;
  - moved all initialization code to utils.c;
  - renamed core/mp_datetime.c to core/datetime.c because it makes more
    sense now;

* Version #1 - initial RFC

In brief
--------

This patchset implements datetime lua support in box, with serialization 
to messagepack, yaml, json and lua mode. Also it contains storage 
engines' indices implementation for datetime type introduced.

* Current implementation is heavily influenced by Sci-Lua lua-time module
  https://github.com/stepelu/lua-time 
  e.g. you could find very similar approach for handling of operations
  with year or month long intervals (which should be handled differently than
  usual intervals of seconds, or days).

* But internally we actually use Christian Hanson' c-dt module
  https://github.com/chansen/c-dt 
  (though it has been modified slightly for cleaner integration
  into our cmake-based build process).


Datetime Module API
-------------------

Small draft of documentation for our `datetime` module API has been
extracted to the discussion topic there - 
https://github.com/tarantool/tarantool/discussions/6244#discussioncomment-1043988
We try to update documentation there once any relevant change introdcued 
to the implementaton.

Messagepack serialization schema
--------------------------------

In short it looks like:
- now we introduce new MP_EXT extension type #4;
- we may save 1 required and 2 optional fields for datetime field;

In all gory details it's explained in MessagePack serialization schema depicted here:
https://github.com/tarantool/tarantool/discussions/6244#discussioncomment-1043990


https://github.com/tarantool/tarantool/issues/5941
https://github.com/tarantool/tarantool/issues/5946

https://github.com/tarantool/tarantool/tree/tsafin/gh-5941-datetime-v5


Timur Safin (5):
  build, lua: built-in module datetime
  box, datetime: messagepack support for datetime
  box, datetime: datetime comparison for indices
  datetime: perf test for datetime parser
  datetime: changelog for datetime module

 .gitmodules                                   |   3 +
 CMakeLists.txt                                |   8 +
 .../gh-5941-datetime-type-support.md          |   6 +
 cmake/BuildCDT.cmake                          |  10 +
 extra/exports                                 |  35 ++
 perf/CMakeLists.txt                           |   3 +
 perf/datetime-common.h                        | 105 ++++
 perf/datetime-compare.cc                      | 213 +++++++
 perf/datetime-parser.cc                       | 105 ++++
 src/CMakeLists.txt                            |   5 +-
 src/box/field_def.c                           |  35 +-
 src/box/field_def.h                           |   1 +
 src/box/lua/serialize_lua.c                   |   8 +-
 src/box/msgpack.c                             |   7 +-
 src/box/tuple_compare.cc                      |  77 +++
 src/lib/core/CMakeLists.txt                   |   5 +-
 src/lib/core/datetime.c                       | 134 +++++
 src/lib/core/datetime.h                       | 121 ++++
 src/lib/core/mp_datetime.c                    | 171 ++++++
 src/lib/core/mp_datetime.h                    |  64 ++
 src/lib/core/mp_extension_types.h             |   1 +
 src/lib/core/mp_utils.h                       |  64 ++
 src/lib/mpstream/mpstream.c                   |  11 +
 src/lib/mpstream/mpstream.h                   |   4 +
 src/lua/datetime.lua                          | 567 ++++++++++++++++++
 src/lua/init.c                                |   4 +-
 src/lua/msgpack.c                             |  12 +
 src/lua/msgpackffi.lua                        |  18 +
 src/lua/serializer.c                          |   4 +
 src/lua/serializer.h                          |   2 +
 src/lua/utils.c                               |  27 +
 src/lua/utils.h                               |  12 +
 test/app-tap/datetime.test.lua                | 213 +++++++
 test/engine/datetime.result                   |  77 +++
 test/engine/datetime.test.lua                 |  35 ++
 test/unit/CMakeLists.txt                      |   3 +-
 test/unit/datetime.c                          | 383 ++++++++++++
 test/unit/datetime.result                     | 471 +++++++++++++++
 third_party/c-dt                              |   1 +
 third_party/lua-cjson/lua_cjson.c             |   8 +
 third_party/lua-yaml/lyaml.cc                 |   6 +-
 41 files changed, 3017 insertions(+), 22 deletions(-)
 create mode 100644 changelogs/unreleased/gh-5941-datetime-type-support.md
 create mode 100644 cmake/BuildCDT.cmake
 create mode 100644 perf/datetime-common.h
 create mode 100644 perf/datetime-compare.cc
 create mode 100644 perf/datetime-parser.cc
 create mode 100644 src/lib/core/datetime.c
 create mode 100644 src/lib/core/datetime.h
 create mode 100644 src/lib/core/mp_datetime.c
 create mode 100644 src/lib/core/mp_datetime.h
 create mode 100644 src/lib/core/mp_utils.h
 create mode 100644 src/lua/datetime.lua
 create mode 100755 test/app-tap/datetime.test.lua
 create mode 100644 test/engine/datetime.result
 create mode 100644 test/engine/datetime.test.lua
 create mode 100644 test/unit/datetime.c
 create mode 100644 test/unit/datetime.result
 create mode 160000 third_party/c-dt

-- 
2.29.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime
  2021-08-19  2:56 [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation Timur Safin via Tarantool-patches
@ 2021-08-19  2:56 ` Timur Safin via Tarantool-patches
  2021-08-19  9:43   ` Serge Petrenko via Tarantool-patches
  2021-08-19 15:26   ` Vladimir Davydov via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 2/5] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
                   ` (3 subsequent siblings)
  4 siblings, 2 replies; 20+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-19  2:56 UTC (permalink / raw)
  To: vdavydov, sergepetrenko, tarantool-patches; +Cc: v.shpilevoy

New third_party module - c-dt
-----------------------------

* Integrated chansen/c-dt parser as 3rd party module to the
  Tarantool cmake build process;
* Points to tarantool/c-dt instead of original chansen/c-dt to
  have easier build integration, because there are additional
  commits which have integrated cmake support and have established
  symbols renaming facilities (similar to those we see in xxhash
  or icu).
* took special care that generated build artifacts not override
  in-source files, but use different build/ directory.

* added datetime parsing unit tests:
  - for literals - with and without trailing timezones;
  - we check that strftime is reversible and produce consistent
    results after roundtrip from/to strings;

New built-in module `datetime`
------------------------------

* created a new Tarantool built-in module `datetime`;
* register new cdef types for this module;
* reexported some `dt_*` functions from `c-dt` library.
  We have to rename externally public symbols to avoid
  name clashes possible usage of this same party module but
  from inside of dynamic libraries. We prefix c-dt symbols
  in the Tarantool build with `tnt_` prefix;

* `strftime` implemented as a simple ffi wrappers in C
  kernel code:

We used to use heavy ffi-related Lua code, but it appears
to be quite fragile, if we run this code on different OS
targets. To make this Lua code less platfrom dependent we
have moved all platform specifics to the C level.
Specifically we want to avoid dependence on `struct tm {}`
layout and GLIBC-specific `strftime` behaviour (which
differs to BSD/Mac LIBC behaviour).

* display datetime

  - introduced output routine for converting datetime
    to their default output format.

  - use this routine for tostring() in datetime.lua

- verify that we parse expected, and fail with unexpected:
  - for that, in the datetime.lua, we have extended api of `parse_date()`,
    `parse_time()`, and `parse_time_zone()` so they return
    not only parsed object, but also a length of parsed substring;
  - which allows us to parse even _partially_ valid strings like
    "20121224 Foo bar".

* simplified interfaces
  - totable() export table values similar to os.date('*t')
  - set() provide unified interface to set values using
    the same set of attributes as in totable()

Part of #5941

@TarantoolBot document
Title: Introduced new built-in `datetime` module

`datetime` module has been introduced, which allows to parse
ISO-8601 literals representing timestamps of various formats,
and then manipulate with date objects.

Please refer to https://github.com/tarantool/tarantool/discussions/6244
for more detailed description of module API.

Part of #5941
---
 .gitmodules                    |   3 +
 CMakeLists.txt                 |   8 +
 cmake/BuildCDT.cmake           |  10 +
 extra/exports                  |  32 ++
 src/CMakeLists.txt             |   5 +-
 src/lib/core/CMakeLists.txt    |   1 +
 src/lib/core/datetime.c        | 125 ++++++++
 src/lib/core/datetime.h        |  86 +++++
 src/lua/datetime.lua           | 567 +++++++++++++++++++++++++++++++++
 src/lua/init.c                 |   4 +-
 src/lua/utils.c                |  27 ++
 src/lua/utils.h                |  12 +
 test/app-tap/datetime.test.lua | 213 +++++++++++++
 test/unit/CMakeLists.txt       |   3 +-
 test/unit/datetime.c           | 260 +++++++++++++++
 test/unit/datetime.result      | 358 +++++++++++++++++++++
 third_party/c-dt               |   1 +
 17 files changed, 1712 insertions(+), 3 deletions(-)
 create mode 100644 cmake/BuildCDT.cmake
 create mode 100644 src/lib/core/datetime.c
 create mode 100644 src/lib/core/datetime.h
 create mode 100644 src/lua/datetime.lua
 create mode 100755 test/app-tap/datetime.test.lua
 create mode 100644 test/unit/datetime.c
 create mode 100644 test/unit/datetime.result
 create mode 160000 third_party/c-dt

diff --git a/.gitmodules b/.gitmodules
index f2f91ee72..aa3fbae4e 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -43,3 +43,6 @@
 [submodule "third_party/xxHash"]
 	path = third_party/xxHash
 	url = https://github.com/tarantool/xxHash
+[submodule "third_party/c-dt"]
+	path = third_party/c-dt
+	url = https://github.com/tarantool/c-dt.git
diff --git a/CMakeLists.txt b/CMakeLists.txt
index e25b81eac..8037c30a7 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -571,6 +571,14 @@ endif()
 # zstd
 #
 
+#
+# Christian Hansen c-dt
+#
+
+include(BuildCDT)
+libccdt_build()
+add_dependencies(build_bundled_libs cdt)
+
 #
 # Third-Party misc
 #
diff --git a/cmake/BuildCDT.cmake b/cmake/BuildCDT.cmake
new file mode 100644
index 000000000..80b26c64a
--- /dev/null
+++ b/cmake/BuildCDT.cmake
@@ -0,0 +1,10 @@
+macro(libccdt_build)
+    set(LIBCDT_INCLUDE_DIRS ${PROJECT_SOURCE_DIR}/third_party/c-dt/)
+    set(LIBCDT_LIBRARIES cdt)
+
+    file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt/build/)
+    add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/c-dt
+                     ${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt/build/)
+    set_target_properties(cdt PROPERTIES COMPILE_FLAGS "-DDT_NAMESPACE=tnt_")
+    add_definitions("-DDT_NAMESPACE=tnt_")
+endmacro()
diff --git a/extra/exports b/extra/exports
index 9eaba1282..0e7392d61 100644
--- a/extra/exports
+++ b/extra/exports
@@ -148,8 +148,16 @@ csv_feed
 csv_iterator_create
 csv_next
 csv_setopt
+datetime_now
+datetime_strftime
+datetime_to_string
+datetime_unpack
 decimal_from_string
 decimal_unpack
+dt_year
+dt_month
+dt_doy
+dt_dom
 error_ref
 error_set_prev
 error_unref
@@ -447,6 +455,30 @@ title_set_status
 title_update
 tnt_default_cert_dir_paths
 tnt_default_cert_file_paths
+tnt_dt_add_years
+tnt_dt_add_quarters
+tnt_dt_add_months
+tnt_dt_dow
+tnt_dt_from_rdn
+tnt_dt_from_struct_tm
+tnt_dt_from_yd
+tnt_dt_from_ymd
+tnt_dt_from_yqd
+tnt_dt_from_ywd
+tnt_dt_parse_iso_date
+tnt_dt_parse_iso_time_basic
+tnt_dt_parse_iso_time_extended
+tnt_dt_parse_iso_time
+tnt_dt_parse_iso_zone_basic
+tnt_dt_parse_iso_zone_extended
+tnt_dt_parse_iso_zone_lenient
+tnt_dt_parse_iso_zone
+tnt_dt_rdn
+tnt_dt_to_struct_tm
+tnt_dt_to_yd
+tnt_dt_to_ymd
+tnt_dt_to_yqd
+tnt_dt_to_ywd
 tnt_iconv
 tnt_iconv_close
 tnt_iconv_open
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index adb03b3f4..4473ff1da 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -51,6 +51,8 @@ lua_source(lua_sources ../third_party/luafun/fun.lua)
 lua_source(lua_sources lua/httpc.lua)
 lua_source(lua_sources lua/iconv.lua)
 lua_source(lua_sources lua/swim.lua)
+lua_source(lua_sources lua/datetime.lua)
+
 # LuaJIT jit.* library
 lua_source(lua_sources ${LUAJIT_SOURCE_ROOT}/src/jit/bc.lua)
 lua_source(lua_sources ${LUAJIT_SOURCE_ROOT}/src/jit/bcsave.lua)
@@ -193,7 +195,8 @@ target_link_libraries(server core coll http_parser bit uri uuid swim swim_udp
 # Rule of thumb: if exporting a symbol from a static library, list the
 # library here.
 set (reexport_libraries server core misc bitset csv swim swim_udp swim_ev
-     shutdown ${LUAJIT_LIBRARIES} ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES} ${CURL_LIBRARIES} ${XXHASH_LIBRARIES})
+     shutdown ${LUAJIT_LIBRARIES} ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES}
+     ${CURL_LIBRARIES} ${XXHASH_LIBRARIES} ${LIBCDT_LIBRARIES})
 
 set (common_libraries
     ${reexport_libraries}
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index 2cd4d0b4f..8bc776b82 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -30,6 +30,7 @@ set(core_sources
     decimal.c
     mp_decimal.c
     cord_buf.c
+    datetime.c
 )
 
 if (TARGET_OS_NETBSD)
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
new file mode 100644
index 000000000..7125090e6
--- /dev/null
+++ b/src/lib/core/datetime.c
@@ -0,0 +1,125 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#include <assert.h>
+#include <limits.h>
+#include <string.h>
+#include <time.h>
+
+#include "trivia/util.h"
+#include "datetime.h"
+
+/*
+ * Given the seconds from Epoch (1970-01-01) we calculate date
+ * since Rata Die (0001-01-01).
+ * DT_EPOCH_1970_OFFSET is the distance in days from Rata Die to Epoch.
+ */
+static int
+local_dt(int64_t secs)
+{
+	return dt_from_rdn((int)(secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET);
+}
+
+static struct tm *
+datetime_to_tm(const struct datetime *date)
+{
+	static struct tm tm;
+
+	memset(&tm, 0, sizeof(tm));
+	int64_t secs = date->secs;
+	dt_to_struct_tm(local_dt(secs), &tm);
+
+	int seconds_of_day = (int64_t)date->secs % SECS_PER_DAY;
+	tm.tm_hour = (seconds_of_day / 3600) % 24;
+	tm.tm_min = (seconds_of_day / 60) % 60;
+	tm.tm_sec = seconds_of_day % 60;
+
+	return &tm;
+}
+
+void
+datetime_now(struct datetime *now)
+{
+	struct timeval tv;
+	gettimeofday(&tv, NULL);
+	now->secs = tv.tv_sec;
+	now->nsec = tv.tv_usec * 1000;
+
+	time_t now_seconds;
+	time(&now_seconds);
+	struct tm tm;
+	localtime_r(&now_seconds, &tm);
+	now->offset = tm.tm_gmtoff / 60;
+}
+
+size_t
+datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
+		  uint32_t len)
+{
+	struct tm *p_tm = datetime_to_tm(date);
+	return strftime(buf, len, fmt, p_tm);
+}
+
+
+/* NB! buf may be NULL, and we should handle it gracefully, returning
+ * calculated length of output string
+ */
+int
+datetime_to_string(const struct datetime *date, char *buf, int len)
+{
+	int offset = date->offset;
+	/* for negative offsets around Epoch date we could get
+	 * negative secs value, which should be attributed to
+	 * 1969-12-31, not 1970-01-01, thus we first shift
+	 * epoch to Rata Die then divide by seconds per day,
+	 * not in reverse
+	 */
+	int64_t rd_seconds = (int64_t)date->secs + offset * 60 +
+			     SECS_EPOCH_1970_OFFSET;
+	int rd_number = rd_seconds / SECS_PER_DAY;
+	assert(rd_number <= INT_MAX);
+	assert(rd_number >= INT_MIN);
+	dt_t dt = dt_from_rdn(rd_number);
+
+	int year, month, day, second, nanosec, sign;
+	dt_to_ymd(dt, &year, &month, &day);
+
+	int hour = (rd_seconds / 3600) % 24;
+	int minute = (rd_seconds / 60) % 60;
+	second = rd_seconds % 60;
+	nanosec = date->nsec;
+
+	int sz = 0;
+	SNPRINT(sz, snprintf, buf, len, "%04d-%02d-%02dT%02d:%02d",
+		year, month, day, hour, minute);
+	if (second || nanosec) {
+		SNPRINT(sz, snprintf, buf, len, ":%02d", second);
+		if (nanosec) {
+			if ((nanosec % 1000000) == 0)
+				SNPRINT(sz, snprintf, buf, len, ".%03d",
+					nanosec / 1000000);
+			else if ((nanosec % 1000) == 0)
+				SNPRINT(sz, snprintf, buf, len, ".%06d",
+					nanosec / 1000);
+			else
+				SNPRINT(sz, snprintf, buf, len, ".%09d", nanosec);
+		}
+	}
+	if (offset == 0) {
+		SNPRINT(sz, snprintf, buf, len, "Z");
+	} else {
+		if (offset < 0) {
+			sign = '-';
+			offset = -offset;
+		} else {
+			sign = '+';
+		}
+		SNPRINT(sz, snprintf, buf, len, "%c%02d:%02d", sign,
+			offset / 60, offset % 60);
+	}
+	return sz;
+}
+
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
new file mode 100644
index 000000000..71feefded
--- /dev/null
+++ b/src/lib/core/datetime.h
@@ -0,0 +1,86 @@
+#pragma once
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#include <stdint.h>
+#include <stdbool.h>
+#include "c-dt/dt.h"
+
+#if defined(__cplusplus)
+extern "C"
+{
+#endif /* defined(__cplusplus) */
+
+/**
+ * We count dates since so called "Rata Die" date
+ * January 1, 0001, Monday (as Day 1).
+ * But datetime structure keeps seconds since
+ * Unix "Epoch" date:
+ * Unix, January 1, 1970, Thursday
+ *
+ * The difference between Epoch (1970-01-01)
+ * and Rata Die (0001-01-01) is 719163 days.
+ */
+
+#ifndef SECS_PER_DAY
+#define SECS_PER_DAY          86400
+#define DT_EPOCH_1970_OFFSET  719163
+#endif
+
+#define SECS_EPOCH_1970_OFFSET 	\
+	((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
+/**
+ * datetime structure keeps number of seconds since
+ * Unix Epoch.
+ * Time is normalized by UTC, so time-zone offset
+ * is informative only.
+ */
+struct datetime {
+	/** Seconds since Epoch. */
+	double secs;
+	/** Nanoseconds, if any. */
+	uint32_t nsec;
+	/** Offset in minutes from UTC. */
+	int32_t offset;
+};
+
+/**
+ * Date/time interval structure
+ */
+struct datetime_interval {
+	/** Relative seconds delta. */
+	double secs;
+	/** Nanoseconds delta, if any. */
+	uint32_t nsec;
+};
+
+/**
+ * Convert datetime to string using default format
+ * @param date source datetime value
+ * @param buf output character buffer
+ * @param len size ofoutput buffer
+ */
+int
+datetime_to_string(const struct datetime *date, char *buf, int len);
+
+/**
+ * Convert datetime to string using default format provided
+ * Wrapper around standard strftime() function
+ * @param date source datetime value
+ * @param fmt format
+ * @param buf output buffer
+ * @param len size of output buffer
+ */
+size_t
+datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
+		  uint32_t len);
+
+void
+datetime_now(struct datetime *now);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
new file mode 100644
index 000000000..88aad0744
--- /dev/null
+++ b/src/lua/datetime.lua
@@ -0,0 +1,567 @@
+local ffi = require('ffi')
+
+--[[
+    `c-dt` library functions handles properly both positive and negative `dt`
+    values, where `dt` is a number of dates since Rata Die date (0001-01-01).
+
+    For better compactness of our typical data in MessagePack stream we shift
+    root of our time to the Unix Epoch date (1970-01-01), thus our 0 is
+    actually dt = 719163.
+
+    So here is a simple formula how convert our epoch-based seconds to dt values
+        dt = (secs / 86400) + 719163
+    Where 719163 is an offset of Unix Epoch (1970-01-01) since Rata Die
+    (0001-01-01) in dates.
+]]
+
+-- dt_core.h definitions
+ffi.cdef [[
+
+typedef int dt_t;
+
+typedef enum {
+    DT_MON       = 1,
+    DT_MONDAY    = 1,
+    DT_TUE       = 2,
+    DT_TUESDAY   = 2,
+    DT_WED       = 3,
+    DT_WEDNESDAY = 3,
+    DT_THU       = 4,
+    DT_THURSDAY  = 4,
+    DT_FRI       = 5,
+    DT_FRIDAY    = 5,
+    DT_SAT       = 6,
+    DT_SATURDAY  = 6,
+    DT_SUN       = 7,
+    DT_SUNDAY    = 7,
+} dt_dow_t;
+
+dt_t   tnt_dt_from_rdn     (int n);
+dt_t   tnt_dt_from_ymd     (int y, int m, int d);
+int    tnt_dt_rdn          (dt_t dt);
+
+dt_dow_t tnt_dt_dow        (dt_t dt);
+
+]]
+
+-- dt_accessor.h
+ffi.cdef [[
+
+int     dt_year         (dt_t dt);
+int     dt_month        (dt_t dt);
+
+int     dt_doy          (dt_t dt);
+int     dt_dom          (dt_t dt);
+
+]]
+
+-- dt_arithmetic.h definitions
+ffi.cdef [[
+
+typedef enum {
+    DT_EXCESS,
+    DT_LIMIT,
+    DT_SNAP
+} dt_adjust_t;
+
+dt_t   tnt_dt_add_years    (dt_t dt, int delta, dt_adjust_t adjust);
+dt_t   tnt_dt_add_quarters (dt_t dt, int delta, dt_adjust_t adjust);
+dt_t   tnt_dt_add_months   (dt_t dt, int delta, dt_adjust_t adjust);
+
+]]
+
+-- dt_parse_iso.h definitions
+ffi.cdef [[
+
+size_t tnt_dt_parse_iso_date (const char *str, size_t len, dt_t *dt);
+size_t tnt_dt_parse_iso_time (const char *str, size_t len, int *sod, int *nsec);
+size_t tnt_dt_parse_iso_zone_lenient(const char *str, size_t len, int *offset);
+
+]]
+
+-- Tarantool functions - datetime.c
+ffi.cdef [[
+
+int    datetime_to_string(const struct datetime * date, char *buf, int len);
+size_t datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
+                         uint32_t len);
+void   datetime_now(struct datetime *now);
+
+]]
+
+local builtin = ffi.C
+local math_modf = math.modf
+local math_floor = math.floor
+
+local SECS_PER_DAY     = 86400
+local NANOS_PER_SEC    = 1000000000
+
+-- c-dt/dt_config.h
+
+-- Unix, January 1, 1970, Thursday
+local DT_EPOCH_1970_OFFSET = 719163
+
+
+local datetime_t = ffi.typeof('struct datetime')
+
+local function is_datetime(o)
+    return ffi.istype(datetime_t, o)
+end
+
+local function check_date(o, message)
+    if not is_datetime(o) then
+        return error(("%s: expected datetime, but received %s"):
+                     format(message, o), 2)
+    end
+end
+
+local function check_str(s, message)
+    if not type(s) == 'string' then
+        return error(("%s: expected string, but received %s"):
+                     format(message, s), 2)
+    end
+end
+
+local function check_range(v, range, txt)
+    assert(#range == 2)
+    if v < range[1] or v > range[2] then
+        error(('value %d of %s is out of allowed range [%d, %d]'):
+              format(v, txt, range[1], range[2]), 4)
+    end
+end
+
+local SECS_EPOCH_OFFSET = (DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
+
+local function local_rd(secs)
+    return math_floor((secs + SECS_EPOCH_OFFSET) / SECS_PER_DAY)
+end
+
+local function local_dt(secs)
+    return builtin.tnt_dt_from_rdn(local_rd(secs))
+end
+
+local function normalize_nsec(secs, nsec)
+    if nsec < 0 then
+        secs = secs - 1
+        nsec = nsec + NANOS_PER_SEC
+    elseif nsec >= NANOS_PER_SEC then
+        secs = secs + 1
+        nsec = nsec - NANOS_PER_SEC
+    end
+    return secs, nsec
+end
+
+local function datetime_cmp(lhs, rhs)
+    if not is_datetime(lhs) or not is_datetime(rhs) then
+        return nil
+    end
+    local sdiff = lhs.secs - rhs.secs
+    return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
+end
+
+local function datetime_eq(lhs, rhs)
+    local rc = datetime_cmp(lhs, rhs)
+    return rc ~= nil and rc == 0
+end
+
+local function datetime_lt(lhs, rhs)
+    local rc = datetime_cmp(lhs, rhs)
+    return rc == nil and error('incompatible types for comparison', 2) or
+           rc < 0
+end
+
+local function datetime_le(lhs, rhs)
+    local rc = datetime_cmp(lhs, rhs)
+    return rc == nil and error('incompatible types for comparison', 2) or
+           rc <= 0
+end
+
+local function datetime_serialize(self)
+    return { secs = self.secs, nsec = self.nsec, offset = self.offset }
+end
+
+local parse_zone
+
+local function datetime_new_raw(secs, nsec, offset)
+    local dt_obj = ffi.new(datetime_t)
+    dt_obj.secs = secs
+    dt_obj.nsec = nsec
+    dt_obj.offset = offset
+    return dt_obj
+end
+
+local function datetime_new_dt(dt, secs, fraction, offset)
+    local epochV = dt ~= nil and (builtin.tnt_dt_rdn(dt) - DT_EPOCH_1970_OFFSET) *
+                   SECS_PER_DAY or 0
+    local secsV = secs or 0
+    local fracV = fraction or 0
+    local ofsV = offset or 0
+    return datetime_new_raw(epochV + secsV - ofsV * 60, fracV, ofsV)
+end
+
+-- create datetime given attribute values from obj
+-- { secs = N, nsec = M, offset = O}
+local function datetime_new_obj(obj, ...)
+    if obj == nil or type(obj) == 'table' then
+        return ffi.new(datetime_t, obj)
+    else
+        return datetime_new_raw(obj, ...)
+    end
+end
+
+-- create datetime given attribute values from obj
+local function datetime_new(obj)
+    if obj == nil or type(obj) ~= 'table' then
+        return datetime_new_raw(0, 0, 0)
+    end
+    local ymd = false
+
+    local nsec = 0
+    local hms = false
+
+    local dt = 0
+
+    local y = obj.year
+    if y ~= nil then
+        check_range(y, {1, 9999}, 'year')
+        ymd = true
+    end
+    local M = obj.month
+    if M ~= nil then
+        check_range(M, {1, 12}, 'month')
+        ymd = true
+    end
+    local d = obj.day
+    if d ~= nil then
+        check_range(d, {1, 31}, 'day')
+        ymd = true
+    end
+    local h = obj.hour
+    if h ~= nil then
+        check_range(h, {0, 23}, 'hour')
+        hms = true
+    end
+    local m = obj.min
+    if m ~= nil then
+        check_range(m, {0, 59}, 'min')
+        hms = true
+    end
+    local ts = obj.sec
+    local s = 0
+    if ts ~= nil then
+        check_range(ts, {0, 60}, 'sec')
+        s, nsec = math_modf(ts)
+        nsec = nsec * 1e9 -- convert fraction to nanoseconds
+        hms = true
+    end
+    local offset = obj.tz
+    if offset ~= nil then
+        if type(offset) == 'number' then
+            -- tz offset in minutes
+            check_range(offset, {0, 720}, offset)
+        elseif type(offset) == 'string' then
+            local zone = parse_zone(offset)
+            if zone == nil then
+                error(('invalid time-zone format %s'):format(offset), 2)
+            else
+                offset = zone.offset
+            end
+        end
+    end
+
+    -- .year, .month, .day
+    if ymd then
+        dt = builtin.tnt_dt_from_ymd(y or 0, M or 0, d or 0)
+    end
+
+    -- .hour, .minute, .second
+    local secs = 0
+    if hms then
+        secs = (h or 0) * 3600 + (m or 0) * 60 + (s or 0)
+    end
+
+    return datetime_new_dt(dt, secs, nsec, offset)
+end
+
+--[[
+    Convert to text datetime values
+
+    - datetime will use ISO-8601 forat:
+        1970-01-01T00:00Z
+        2021-08-18T16:57:08.981725+03:00
+]]
+local function datetime_tostring(o)
+    if ffi.typeof(o) == datetime_t then
+        local sz = 48
+        local buff = ffi.new('char[?]', sz)
+        local len = builtin.datetime_to_string(o, buff, sz)
+        assert(len < sz)
+        return ffi.string(buff)
+    end
+end
+
+--[[
+    Parse partial ISO-8601 date string
+
+    Accepetd formats are:
+
+    Basic      Extended
+    20121224   2012-12-24   Calendar date   (ISO 8601)
+    2012359    2012-359     Ordinal date    (ISO 8601)
+    2012W521   2012-W52-1   Week date       (ISO 8601)
+    2012Q485   2012-Q4-85   Quarter date
+
+    Returns pair of constructed datetime object, and length of string
+    which has been accepted by parser.
+]]
+
+local function parse_date(str)
+    check_str("datetime.parse_date()")
+    local dt = ffi.new('dt_t[1]')
+    local len = builtin.tnt_dt_parse_iso_date(str, #str, dt)
+    return len > 0 and datetime_new_dt(dt[0]) or nil, tonumber(len)
+end
+
+--[[
+    Basic    Extended
+    Z        N/A
+    +hh      N/A
+    -hh      N/A
+    +hhmm    +hh:mm
+    -hhmm    -hh:mm
+
+    Returns pair of constructed datetime object, and length of string
+    which has been accepted by parser.
+]]
+parse_zone = function(str)
+    check_str("datetime.parse_zone()")
+    local offset = ffi.new('int[1]')
+    local len = builtin.tnt_dt_parse_iso_zone_lenient(str, #str, offset)
+    return len > 0 and offset[0], tonumber(len)
+end
+
+--[[
+    aggregated parse functions
+    assumes to deal with date T time time_zone
+    at once
+
+    date [T] time [ ] time_zone
+
+    Returns constructed datetime object.
+]]
+local function parse(str)
+    check_str("datetime.parse()")
+    local dt = ffi.new('dt_t[1]')
+    local len = #str
+    local n = builtin.tnt_dt_parse_iso_date(str, len, dt)
+    local dt_ = dt[0]
+    if n == 0 or len == n then
+        return datetime_new_dt(dt_)
+    end
+
+    str = str:sub(tonumber(n) + 1)
+
+    local ch = str:sub(1, 1)
+    if ch:match('[Tt ]') == nil then
+        return datetime_new_dt(dt_)
+    end
+
+    str = str:sub(2)
+    len = #str
+
+    local sp = ffi.new('int[1]')
+    local fp = ffi.new('int[1]')
+    local n = builtin.tnt_dt_parse_iso_time(str, len, sp, fp)
+    if n == 0 then
+        return datetime_new_dt(dt_)
+    end
+    local sp_ = sp[0]
+    local fp_ = fp[0]
+    if len == n then
+        return datetime_new_dt(dt_, sp_, fp_)
+    end
+
+    str = str:sub(tonumber(n) + 1)
+
+    if str:sub(1, 1) == ' ' then
+        str = str:sub(2)
+    end
+
+    len = #str
+
+    local offset = ffi.new('int[1]')
+    n = builtin.tnt_dt_parse_iso_zone_lenient(str, len, offset)
+    if n == 0 then
+        return datetime_new_dt(dt_, sp_, fp_)
+    end
+    return datetime_new_dt(dt_, sp_, fp_, offset[0])
+end
+
+--[[
+    Dispatch function to create datetime from string or table.
+    Creates default timeobject (pointing to Epoch date) if
+    called without arguments.
+]]
+local function datetime_from(o)
+    if o == nil or type(o) == 'table' then
+        return datetime_new(o)
+    elseif type(o) == 'string' then
+        return parse(o)
+    end
+end
+
+--[[
+    Create datetime object representing current time using microseconds
+    platform timer and local timezone information.
+]]
+local function local_now()
+    local d = datetime_new_raw(0, 0, 0)
+    builtin.datetime_now(d)
+    return d
+end
+
+-- addition or subtraction from date/time of a given interval
+-- described via table direction should be +1 or -1
+local function datetime_increment(self, o, direction)
+    assert(direction == -1 or direction == 1)
+    local title = direction > 0 and "datetime.add" or "datetime.sub"
+    check_date(self, title)
+    if type(o) ~= 'table' then
+        error(('%s - object expected'):format(title), 2)
+    end
+
+    local secs, nsec = self.secs, self.nsec
+    local offset = self.offset
+
+    -- operations with intervals should be done using human dates
+    -- not UTC dates, thus we normalize to UTC
+    local dt = local_dt(secs)
+
+    local ym_updated = false
+    local years, months, weeks = o.years, o.months, o.weeks
+
+    if years ~= nil then
+        check_range(years, {0, 9999}, 'years')
+        dt = builtin.tnt_dt_add_years(dt, direction * years, builtin.DT_LIMIT)
+        ym_updated = true
+    end
+    if months ~= nil then
+        check_range(months, {0, 12}, 'months')
+        dt = builtin.tnt_dt_add_months(dt, direction * months, builtin.DT_LIMIT)
+        ym_updated = true
+    end
+    if ym_updated then
+        secs = (builtin.tnt_dt_rdn(dt) - DT_EPOCH_1970_OFFSET) * SECS_PER_DAY +
+                secs % SECS_PER_DAY
+    end
+
+    if weeks ~= nil then
+        check_range(weeks, {0, 52}, 'weeks')
+        secs = secs + direction * 7 * weeks * SECS_PER_DAY
+    end
+
+    local days, hours, minutes, seconds = o.days, o.hours, o.minutes, o.seconds
+    if days ~= nil then
+        check_range(days, {0, 31}, 'days')
+        secs = secs + direction * days * SECS_PER_DAY
+    end
+    if hours ~= nil then
+        check_range(hours, {0, 23}, 'hours')
+        secs = secs + direction * 60 * 60 * hours
+    end
+    if minutes ~= nil then
+        check_range(minutes, {0, 59}, 'minutes')
+        secs = secs + direction * 60 * minutes
+    end
+    if seconds ~= nil then
+        check_range(seconds, {0, 60}, 'seconds')
+        local s, frac = math.modf(seconds)
+        secs = secs + direction * s
+        nsec = nsec + direction * frac * 1e9
+    end
+
+    secs, nsec = normalize_nsec(secs, nsec)
+
+    return datetime_new_raw(secs, nsec, offset)
+end
+
+--[[
+    Return table in os.date('*t') format, but with timezone
+    and nanoseconds
+]]
+local function datetime_totable(self)
+    local secs = self.secs
+    local dt = local_dt(secs)
+    local year = builtin.dt_year(dt)
+    local month = builtin.dt_month(dt)
+    local yday = builtin.dt_doy(dt)
+    local wday = ffi.cast('int32_t', builtin.tnt_dt_dow(dt))
+    local day_of_month = builtin.dt_dom(dt)
+    local hour = math_floor((secs / 3600) % 24)
+    local minute = math_floor((secs / 60) % 60)
+    local second = secs % 60
+
+    return {
+        sec = second,
+        min = minute,
+        day = day_of_month,
+        isdst = false,
+        wday = wday,
+        yday = yday,
+        year = year,
+        month = month,
+        hour = hour,
+        nsec = self.nsec,
+        tz = self.offset,
+    }
+end
+
+local function datetime_set(self, obj)
+    -- FIXME
+    return datetime_new(obj)
+end
+
+local function strftime(fmt, o)
+    check_date(o, "datetime.strftime()")
+    local sz = 128
+    local buff = ffi.new('char[?]', sz)
+    builtin.datetime_strftime(o, fmt, buff, sz)
+    return ffi.string(buff)
+end
+
+ffi.metatype(datetime_t, {
+    __tostring = datetime_tostring,
+    __serialize = datetime_serialize,
+    __eq = datetime_eq,
+    __lt = datetime_lt,
+    __le = datetime_le,
+    __index = {
+        epoch = function(self) return self.secs end,
+        timestamp = function(self) return self.secs + self.nsec / 1e9 end,
+        nanoseconds = function(self) return self.secs * 1e9 + self.nsec end,
+        microseconds = function(self) return self.secs * 1e6 + self.nsec / 1e3 end,
+        milliseconds = function(self) return self.secs * 1e3 + self.nsec / 1e6 end,
+        seconds = function(self) return self.secs + self.nsec / 1e9 end,
+        add = function(self, obj) return datetime_increment(self, obj, 1) end,
+        sub = function(self, obj) return datetime_increment(self, obj, -1) end,
+        totable = datetime_totable,
+        set = datetime_set,
+    }
+})
+
+return setmetatable(
+    {
+        new         = datetime_new,
+        new_raw     = datetime_new_obj,
+
+        parse       = parse,
+        parse_date  = parse_date,
+
+        now         = local_now,
+        strftime    = strftime,
+
+        is_datetime = is_datetime,
+    }, {
+        __call = function(self, ...) return datetime_from(...) end
+    }
+)
diff --git a/src/lua/init.c b/src/lua/init.c
index f9738025d..127e935d7 100644
--- a/src/lua/init.c
+++ b/src/lua/init.c
@@ -129,7 +129,8 @@ extern char strict_lua[],
 	parse_lua[],
 	process_lua[],
 	humanize_lua[],
-	memprof_lua[]
+	memprof_lua[],
+	datetime_lua[]
 ;
 
 static const char *lua_modules[] = {
@@ -184,6 +185,7 @@ static const char *lua_modules[] = {
 	"memprof.process", process_lua,
 	"memprof.humanize", humanize_lua,
 	"memprof", memprof_lua,
+	"datetime", datetime_lua,
 	NULL
 };
 
diff --git a/src/lua/utils.c b/src/lua/utils.c
index c71cd4857..2c89326f3 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -48,6 +48,9 @@ static uint32_t CTID_STRUCT_IBUF_PTR;
 uint32_t CTID_CHAR_PTR;
 uint32_t CTID_CONST_CHAR_PTR;
 uint32_t CTID_UUID;
+uint32_t CTID_DATETIME = 0;
+uint32_t CTID_INTERVAL = 0;
+
 
 void *
 luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
@@ -120,6 +123,12 @@ luaL_pushuuidstr(struct lua_State *L, const struct tt_uuid *uuid)
 	lua_pushlstring(L, str, UUID_STR_LEN);
 }
 
+struct datetime *
+luaL_pushdatetime(struct lua_State *L)
+{
+	return luaL_pushcdata(L, CTID_DATETIME);
+}
+
 int
 luaL_iscdata(struct lua_State *L, int idx)
 {
@@ -725,6 +734,24 @@ tarantool_lua_utils_init(struct lua_State *L)
 	CTID_UUID = luaL_ctypeid(L, "struct tt_uuid");
 	assert(CTID_UUID != 0);
 
+	rc = luaL_cdef(L, "struct datetime {"
+			  "double secs;"
+			  "int32_t nsec;"
+			  "int32_t offset;"
+			  "};");
+	assert(rc == 0);
+	(void) rc;
+	CTID_DATETIME = luaL_ctypeid(L, "struct datetime");
+	assert(CTID_DATETIME != 0);
+	rc = luaL_cdef(L, "struct datetime_interval {"
+			  "double secs;"
+			  "int32_t nsec;"
+			  "};");
+	assert(rc == 0);
+	(void) rc;
+	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval");
+	assert(CTID_INTERVAL != 0);
+
 	lua_pushcfunction(L, luaT_newthread_wrapper);
 	luaT_newthread_ref = luaL_ref(L, LUA_REGISTRYINDEX);
 	return 0;
diff --git a/src/lua/utils.h b/src/lua/utils.h
index 45070b778..73495b607 100644
--- a/src/lua/utils.h
+++ b/src/lua/utils.h
@@ -59,6 +59,7 @@ struct lua_State;
 struct ibuf;
 typedef struct ibuf box_ibuf_t;
 struct tt_uuid;
+struct datetime;
 
 /**
  * Single global lua_State shared by core and modules.
@@ -71,6 +72,8 @@ extern struct lua_State *tarantool_L;
 extern uint32_t CTID_CHAR_PTR;
 extern uint32_t CTID_CONST_CHAR_PTR;
 extern uint32_t CTID_UUID;
+extern uint32_t CTID_DATETIME;
+extern uint32_t CTID_INTERVAL;
 
 struct tt_uuid *
 luaL_pushuuid(struct lua_State *L);
@@ -78,6 +81,15 @@ luaL_pushuuid(struct lua_State *L);
 void
 luaL_pushuuidstr(struct lua_State *L, const struct tt_uuid *uuid);
 
+/**
+ * @brief Push cdata of a datetime type onto the stack.
+ * @param L Lua State
+ * @sa luaL_pushcdata
+ * @return memory associated with this datetime data
+ */
+struct datetime *
+luaL_pushdatetime(struct lua_State *L);
+
 /** \cond public */
 
 /**
diff --git a/test/app-tap/datetime.test.lua b/test/app-tap/datetime.test.lua
new file mode 100755
index 000000000..5c39feee8
--- /dev/null
+++ b/test/app-tap/datetime.test.lua
@@ -0,0 +1,213 @@
+#!/usr/bin/env tarantool
+
+local tap = require('tap')
+local test = tap.test("errno")
+local date = require('datetime')
+local ffi = require('ffi')
+
+test:plan(7)
+
+test:test("Simple tests for parser", function(test)
+    test:plan(2)
+    test:ok(date("1970-01-01T01:00:00Z") ==
+            date {year=1970, month=1, day=1, hour=1, minute=0, second=0})
+    test:ok(date("1970-01-01T02:00:00+02:00") ==
+            date {year=1970, month=1, day=1, hour=2, minute=0, second=0, tz=120})
+end)
+
+test:test("Multiple tests for parser (with nanoseconds)", function(test)
+    test:plan(193)
+    -- borrowed from p5-time-moments/t/180_from_string.t
+    local tests =
+    {
+        {'1970-01-01T00:00Z',                  0,         0,    0, 1},
+        {'1970-01-01T02:00+02:00',             0,         0,  120, 1},
+        {'1970-01-01T01:30+01:30',             0,         0,   90, 1},
+        {'1970-01-01T01:00+01:00',             0,         0,   60, 1},
+        {'1970-01-01T00:01+00:01',             0,         0,    1, 1},
+        {'1970-01-01T00:00Z',                  0,         0,    0, 1},
+        {'1969-12-31T23:59-00:01',             0,         0,   -1, 1},
+        {'1969-12-31T23:00-01:00',             0,         0,  -60, 1},
+        {'1969-12-31T22:30-01:30',             0,         0,  -90, 1},
+        {'1969-12-31T22:00-02:00',             0,         0, -120, 1},
+        {'1970-01-01T00:00:00.123456789Z',     0, 123456789,    0, 1},
+        {'1970-01-01T00:00:00.12345678Z',      0, 123456780,    0, 0},
+        {'1970-01-01T00:00:00.1234567Z',       0, 123456700,    0, 0},
+        {'1970-01-01T00:00:00.123456Z',        0, 123456000,    0, 1},
+        {'1970-01-01T00:00:00.12345Z',         0, 123450000,    0, 0},
+        {'1970-01-01T00:00:00.1234Z',          0, 123400000,    0, 0},
+        {'1970-01-01T00:00:00.123Z',           0, 123000000,    0, 1},
+        {'1970-01-01T00:00:00.12Z',            0, 120000000,    0, 0},
+        {'1970-01-01T00:00:00.1Z',             0, 100000000,    0, 0},
+        {'1970-01-01T00:00:00.01Z',            0,  10000000,    0, 0},
+        {'1970-01-01T00:00:00.001Z',           0,   1000000,    0, 1},
+        {'1970-01-01T00:00:00.0001Z',          0,    100000,    0, 0},
+        {'1970-01-01T00:00:00.00001Z',         0,     10000,    0, 0},
+        {'1970-01-01T00:00:00.000001Z',        0,      1000,    0, 1},
+        {'1970-01-01T00:00:00.0000001Z',       0,       100,    0, 0},
+        {'1970-01-01T00:00:00.00000001Z',      0,        10,    0, 0},
+        {'1970-01-01T00:00:00.000000001Z',     0,         1,    0, 1},
+        {'1970-01-01T00:00:00.000000009Z',     0,         9,    0, 1},
+        {'1970-01-01T00:00:00.00000009Z',      0,        90,    0, 0},
+        {'1970-01-01T00:00:00.0000009Z',       0,       900,    0, 0},
+        {'1970-01-01T00:00:00.000009Z',        0,      9000,    0, 1},
+        {'1970-01-01T00:00:00.00009Z',         0,     90000,    0, 0},
+        {'1970-01-01T00:00:00.0009Z',          0,    900000,    0, 0},
+        {'1970-01-01T00:00:00.009Z',           0,   9000000,    0, 1},
+        {'1970-01-01T00:00:00.09Z',            0,  90000000,    0, 0},
+        {'1970-01-01T00:00:00.9Z',             0, 900000000,    0, 0},
+        {'1970-01-01T00:00:00.99Z',            0, 990000000,    0, 0},
+        {'1970-01-01T00:00:00.999Z',           0, 999000000,    0, 1},
+        {'1970-01-01T00:00:00.9999Z',          0, 999900000,    0, 0},
+        {'1970-01-01T00:00:00.99999Z',         0, 999990000,    0, 0},
+        {'1970-01-01T00:00:00.999999Z',        0, 999999000,    0, 1},
+        {'1970-01-01T00:00:00.9999999Z',       0, 999999900,    0, 0},
+        {'1970-01-01T00:00:00.99999999Z',      0, 999999990,    0, 0},
+        {'1970-01-01T00:00:00.999999999Z',     0, 999999999,    0, 1},
+        {'1970-01-01T00:00:00.0Z',             0,         0,    0, 0},
+        {'1970-01-01T00:00:00.00Z',            0,         0,    0, 0},
+        {'1970-01-01T00:00:00.000Z',           0,         0,    0, 0},
+        {'1970-01-01T00:00:00.0000Z',          0,         0,    0, 0},
+        {'1970-01-01T00:00:00.00000Z',         0,         0,    0, 0},
+        {'1970-01-01T00:00:00.000000Z',        0,         0,    0, 0},
+        {'1970-01-01T00:00:00.0000000Z',       0,         0,    0, 0},
+        {'1970-01-01T00:00:00.00000000Z',      0,         0,    0, 0},
+        {'1970-01-01T00:00:00.000000000Z',     0,         0,    0, 0},
+        {'1973-11-29T21:33:09Z',       123456789,         0,    0, 1},
+        {'2013-10-28T17:51:56Z',      1382982716,         0,    0, 1},
+        {'9999-12-31T23:59:59Z',    253402300799,         0,    0, 1},
+    }
+    for _, value in ipairs(tests) do
+        local str, epoch, nsec, offset, check
+        str, epoch, nsec, offset, check = unpack(value)
+        local dt = date(str)
+        test:ok(dt.secs == epoch, ('%s: dt.secs == %d'):format(str, epoch))
+        test:ok(dt.nsec == nsec, ('%s: dt.nsec == %d'):format(str, nsec))
+        test:ok(dt.offset == offset, ('%s: dt.offset == %d'):format(str, offset))
+        if check > 0 then
+            test:ok(str == tostring(dt), ('%s == tostring(%s)'):
+                    format(str, tostring(dt)))
+        end
+    end
+end)
+
+ffi.cdef [[
+    void tzset(void);
+]]
+
+test:test("Datetime string formatting", function(test)
+    test:plan(5)
+    local str = "1970-01-01"
+    local t = date(str)
+    test:ok(t.secs == 0, ('%s: t.secs == %d'):format(str, tonumber(t.secs)))
+    test:ok(t.nsec == 0, ('%s: t.nsec == %d'):format(str, t.nsec))
+    test:ok(t.offset == 0, ('%s: t.offset == %d'):format(str, t.offset))
+    test:ok(date.strftime('%d/%m/%Y', t) == '01/01/1970', ('%s: strftime #1'):format(str))
+    test:ok(date.strftime('%A %d. %B %Y', t) == 'Thursday 01. January 1970', ('%s: strftime #2'):format(str))
+end)
+
+test:test("Parse iso date - valid strings", function(test)
+    test:plan(32)
+    local good = {
+        {2012, 12, 24, "20121224",                   8 },
+        {2012, 12, 24, "20121224  Foo bar",          8 },
+        {2012, 12, 24, "2012-12-24",                10 },
+        {2012, 12, 24, "2012-12-24 23:59:59",       10 },
+        {2012, 12, 24, "2012-12-24T00:00:00+00:00", 10 },
+        {2012, 12, 24, "2012359",                    7 },
+        {2012, 12, 24, "2012359T235959+0130",        7 },
+        {2012, 12, 24, "2012-359",                   8 },
+        {2012, 12, 24, "2012W521",                   8 },
+        {2012, 12, 24, "2012-W52-1",                10 },
+        {2012, 12, 24, "2012Q485",                   8 },
+        {2012, 12, 24, "2012-Q4-85",                10 },
+        {   1,  1,  1, "0001-Q1-01",                10 },
+        {   1,  1,  1, "0001-W01-1",                10 },
+        {   1,  1,  1, "0001-01-01",                10 },
+        {   1,  1,  1, "0001-001",                   8 },
+    }
+
+    for _, value in ipairs(good) do
+        local year, month, day, str, date_part_len
+        year, month, day, str, date_part_len = unpack(value)
+        local expected_date = date{year = year, month = month, day = day}
+        local date_part, len
+        date_part, len = date.parse_date(str)
+        test:ok(len == date_part_len, ('%s: length check %d'):format(str, len))
+        test:ok(expected_date == date_part, ('%s: expected date'):format(str))
+    end
+end)
+
+test:test("Parse iso date - invalid strings", function(test)
+    test:plan(62)
+    local bad = {
+        "20121232"   , -- Invalid day of month
+        "2012-12-310", -- Invalid day of month
+        "2012-13-24" , -- Invalid month
+        "2012367"    , -- Invalid day of year
+        "2012-000"   , -- Invalid day of year
+        "2012W533"   , -- Invalid week of year
+        "2012-W52-8" , -- Invalid day of week
+        "2012Q495"   , -- Invalid day of quarter
+        "2012-Q5-85" , -- Invalid quarter
+        "20123670"   , -- Trailing digit
+        "201212320"  , -- Trailing digit
+        "2012-12"    , -- Reduced accuracy
+        "2012-Q4"    , -- Reduced accuracy
+        "2012-Q42"   , -- Invalid
+        "2012-Q1-1"  , -- Invalid day of quarter
+        "2012Q--420" , -- Invalid
+        "2012-Q-420" , -- Invalid
+        "2012Q11"    , -- Incomplete
+        "2012Q1234"  , -- Trailing digit
+        "2012W12"    , -- Incomplete
+        "2012W1234"  , -- Trailing digit
+        "2012W-123"  , -- Invalid
+        "2012-W12"   , -- Incomplete
+        "2012-W12-12", -- Trailing digit
+        "2012U1234"  , -- Invalid
+        "2012-1234"  , -- Invalid
+        "2012-X1234" , -- Invalid
+        "0000-Q1-01" , -- Year less than 0001
+        "0000-W01-1" , -- Year less than 0001
+        "0000-01-01" , -- Year less than 0001
+        "0000-001"   , -- Year less than 0001
+    }
+
+    for _, str in ipairs(bad) do
+        local date_part, len
+        date_part, len = date.parse_date(str)
+        test:ok(len == 0, ('%s: length check %d'):format(str, len))
+        test:ok(date_part == nil, ('%s: empty date check %s'):format(str, date_part))
+    end
+end)
+
+test:test("Parse tiny date into seconds and other parts", function(test)
+    test:plan(7)
+    local str = '19700101 00:00:30.528'
+    local tiny = date(str)
+    test:ok(tiny.secs == 30, ("secs of '%s'"):format(str))
+    test:ok(tiny.nsec == 528000000, ("nsec of '%s'"):format(str))
+    test:ok(tiny:nanoseconds() == 30528000000, "nanoseconds")
+    test:ok(tiny:microseconds() == 30528000, "microseconds")
+    test:ok(tiny:milliseconds() == 30528, "milliseconds")
+    test:ok(tiny:seconds() == 30.528, "seconds")
+    test:ok(tiny:timestamp() == 30.528, "timestamp")
+end)
+
+test:test("Time interval operations", function(test)
+    test:plan(2)
+
+    -- check arithmetic with leap dates
+    local T = date('1972-02-29')
+    test:ok(tostring(T:add{years = 1, months = 2}) == '1973-05-01T00:00Z',
+            ('T:add{years=1,months=2}(%s)'):format(T))
+
+    -- check average, not leap dates
+    T = date('1970-01-08')
+    test:ok(tostring(T:add{years = 1, months = 2}) == '1971-03-08T00:00Z',
+            ('T:add{years=1,months=2}(%s)'):format(T))
+
+end)
+
+os.exit(test:check() and 0 or 1)
diff --git a/test/unit/CMakeLists.txt b/test/unit/CMakeLists.txt
index 5bb7cd6e7..8194a133f 100644
--- a/test/unit/CMakeLists.txt
+++ b/test/unit/CMakeLists.txt
@@ -56,7 +56,8 @@ add_executable(uuid.test uuid.c core_test_utils.c)
 target_link_libraries(uuid.test uuid unit)
 add_executable(random.test random.c core_test_utils.c)
 target_link_libraries(random.test core unit)
-
+add_executable(datetime.test datetime.c)
+target_link_libraries(datetime.test cdt core unit)
 add_executable(bps_tree.test bps_tree.cc)
 target_link_libraries(bps_tree.test small misc)
 add_executable(bps_tree_iterator.test bps_tree_iterator.cc)
diff --git a/test/unit/datetime.c b/test/unit/datetime.c
new file mode 100644
index 000000000..931636172
--- /dev/null
+++ b/test/unit/datetime.c
@@ -0,0 +1,260 @@
+#include "dt.h"
+#include <assert.h>
+#include <stdint.h>
+#include <string.h>
+#include <time.h>
+
+#include "unit.h"
+#include "datetime.h"
+#include "trivia/util.h"
+
+static const char sample[] = "2012-12-24T15:30Z";
+
+#define S(s) {s, sizeof(s) - 1}
+struct {
+	const char *str;
+	size_t len;
+} tests[] = {
+	S("2012-12-24 15:30Z"),
+	S("2012-12-24 15:30z"),
+	S("2012-12-24 15:30"),
+	S("2012-12-24 16:30+01:00"),
+	S("2012-12-24 16:30+0100"),
+	S("2012-12-24 16:30+01"),
+	S("2012-12-24 14:30-01:00"),
+	S("2012-12-24 14:30-0100"),
+	S("2012-12-24 14:30-01"),
+	S("2012-12-24 15:30:00Z"),
+	S("2012-12-24 15:30:00z"),
+	S("2012-12-24 15:30:00"),
+	S("2012-12-24 16:30:00+01:00"),
+	S("2012-12-24 16:30:00+0100"),
+	S("2012-12-24 14:30:00-01:00"),
+	S("2012-12-24 14:30:00-0100"),
+	S("2012-12-24 15:30:00.123456Z"),
+	S("2012-12-24 15:30:00.123456z"),
+	S("2012-12-24 15:30:00.123456"),
+	S("2012-12-24 16:30:00.123456+01:00"),
+	S("2012-12-24 16:30:00.123456+01"),
+	S("2012-12-24 14:30:00.123456-01:00"),
+	S("2012-12-24 14:30:00.123456-01"),
+	S("2012-12-24t15:30Z"),
+	S("2012-12-24t15:30z"),
+	S("2012-12-24t15:30"),
+	S("2012-12-24t16:30+01:00"),
+	S("2012-12-24t16:30+0100"),
+	S("2012-12-24t14:30-01:00"),
+	S("2012-12-24t14:30-0100"),
+	S("2012-12-24t15:30:00Z"),
+	S("2012-12-24t15:30:00z"),
+	S("2012-12-24t15:30:00"),
+	S("2012-12-24t16:30:00+01:00"),
+	S("2012-12-24t16:30:00+0100"),
+	S("2012-12-24t14:30:00-01:00"),
+	S("2012-12-24t14:30:00-0100"),
+	S("2012-12-24t15:30:00.123456Z"),
+	S("2012-12-24t15:30:00.123456z"),
+	S("2012-12-24t16:30:00.123456+01:00"),
+	S("2012-12-24t14:30:00.123456-01:00"),
+	S("2012-12-24 16:30 +01:00"),
+	S("2012-12-24 14:30 -01:00"),
+	S("2012-12-24 15:30 UTC"),
+	S("2012-12-24 16:30 UTC+1"),
+	S("2012-12-24 16:30 UTC+01"),
+	S("2012-12-24 16:30 UTC+0100"),
+	S("2012-12-24 16:30 UTC+01:00"),
+	S("2012-12-24 14:30 UTC-1"),
+	S("2012-12-24 14:30 UTC-01"),
+	S("2012-12-24 14:30 UTC-01:00"),
+	S("2012-12-24 14:30 UTC-0100"),
+	S("2012-12-24 15:30 GMT"),
+	S("2012-12-24 16:30 GMT+1"),
+	S("2012-12-24 16:30 GMT+01"),
+	S("2012-12-24 16:30 GMT+0100"),
+	S("2012-12-24 16:30 GMT+01:00"),
+	S("2012-12-24 14:30 GMT-1"),
+	S("2012-12-24 14:30 GMT-01"),
+	S("2012-12-24 14:30 GMT-01:00"),
+	S("2012-12-24 14:30 GMT-0100"),
+	S("2012-12-24 14:30 -01:00"),
+	S("2012-12-24 16:30:00 +01:00"),
+	S("2012-12-24 14:30:00 -01:00"),
+	S("2012-12-24 16:30:00.123456 +01:00"),
+	S("2012-12-24 14:30:00.123456 -01:00"),
+	S("2012-12-24 15:30:00.123456 -00:00"),
+	S("20121224T1630+01:00"),
+	S("2012-12-24T1630+01:00"),
+	S("20121224T16:30+01"),
+	S("20121224T16:30 +01"),
+};
+#undef S
+
+static int
+parse_datetime(const char *str, size_t len, int64_t *secs_p,
+	       int32_t *nanosecs_p, int32_t *offset_p)
+{
+	size_t n;
+	dt_t dt;
+	char c;
+	int sec_of_day = 0, nanosecond = 0, offset = 0;
+
+	n = dt_parse_iso_date(str, len, &dt);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	c = str[n++];
+	if (!(c == 'T' || c == 't' || c == ' '))
+		return 1;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_time(str, len, &sec_of_day, &nanosecond);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	if (str[n] == ' ')
+		n++;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_zone_lenient(str, len, &offset);
+	if (!n || n != len)
+		return 1;
+
+exit:
+	*secs_p = ((int64_t)dt_rdn(dt) - DT_EPOCH_1970_OFFSET) * SECS_PER_DAY +
+		  sec_of_day - offset * 60;
+	*nanosecs_p = nanosecond;
+	*offset_p = offset;
+
+	return 0;
+}
+
+static int
+local_rd(const struct datetime *dt)
+{
+	return (int)((int64_t)dt->secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET;
+}
+
+static int
+local_dt(const struct datetime *dt)
+{
+	return dt_from_rdn(local_rd(dt));
+}
+
+struct tm *
+datetime_to_tm(struct datetime *dt)
+{
+	static struct tm tm;
+
+	memset(&tm, 0, sizeof(tm));
+	dt_to_struct_tm(local_dt(dt), &tm);
+
+	int seconds_of_day = (int64_t)dt->secs % 86400;
+	tm.tm_hour = (seconds_of_day / 3600) % 24;
+	tm.tm_min = (seconds_of_day / 60) % 60;
+	tm.tm_sec = seconds_of_day % 60;
+
+	return &tm;
+}
+
+static void datetime_test(void)
+{
+	size_t index;
+	int64_t secs_expected;
+	int32_t nanosecs;
+	int32_t offset;
+
+	plan(355);
+	parse_datetime(sample, sizeof(sample) - 1,
+		       &secs_expected, &nanosecs, &offset);
+
+	for (index = 0; index < lengthof(tests); index++) {
+		int64_t secs;
+		int rc = parse_datetime(tests[index].str, tests[index].len,
+					&secs, &nanosecs, &offset);
+		is(rc, 0, "correct parse_datetime return value for '%s'",
+		   tests[index].str);
+		is(secs, secs_expected, "correct parse_datetime output "
+					"seconds for '%s",
+		   tests[index].str);
+
+		/*
+		 * check that stringized literal produces the same date
+		 * time fields
+		 */
+		static char buff[40];
+		struct datetime dt = {secs, nanosecs, offset};
+		/* datetime_to_tm returns time in GMT zone */
+		struct tm *p_tm = datetime_to_tm(&dt);
+		size_t len = strftime(buff, sizeof(buff), "%F %T", p_tm);
+		ok(len > 0, "strftime");
+		int64_t parsed_secs;
+		int32_t parsed_nsecs, parsed_ofs;
+		rc = parse_datetime(buff, len, &parsed_secs, &parsed_nsecs, &parsed_ofs);
+		is(rc, 0, "correct parse_datetime return value for '%s'", buff);
+		is(secs, parsed_secs,
+		   "reversible seconds via strftime for '%s", buff);
+	}
+	check_plan();
+}
+
+
+static void
+tostring_datetime_test(void)
+{
+	static struct {
+		const char *string;
+		int64_t     secs;
+		uint32_t    nsec;
+		uint32_t    offset;
+	} tests[] = {
+		{"1970-01-01T02:00+02:00",          0,         0,  120},
+		{"1970-01-01T01:30+01:30",          0,         0,   90},
+		{"1970-01-01T01:00+01:00",          0,         0,   60},
+		{"1970-01-01T00:01+00:01",          0,         0,    1},
+		{"1970-01-01T00:00Z",               0,         0,    0},
+		{"1969-12-31T23:59-00:01",          0,         0,   -1},
+		{"1969-12-31T23:00-01:00",          0,         0,  -60},
+		{"1969-12-31T22:30-01:30",          0,         0,  -90},
+		{"1969-12-31T22:00-02:00",          0,         0, -120},
+		{"1970-01-01T00:00:00.123456789Z",  0, 123456789,    0},
+		{"1970-01-01T00:00:00.123456Z",     0, 123456000,    0},
+		{"1970-01-01T00:00:00.123Z",        0, 123000000,    0},
+		{"1973-11-29T21:33:09Z",    123456789,         0,    0},
+		{"2013-10-28T17:51:56Z",   1382982716,         0,    0},
+		{"9999-12-31T23:59:59Z", 253402300799,         0,    0},
+	};
+	size_t index;
+
+	plan(15);
+	for (index = 0; index < lengthof(tests); index++) {
+		struct datetime date = {
+			tests[index].secs,
+			tests[index].nsec,
+			tests[index].offset
+		};
+		char buf[48];
+		datetime_to_string(&date, buf, sizeof(buf));
+		is(strcmp(buf, tests[index].string), 0,
+		   "string '%s' expected, received '%s'",
+		   tests[index].string, buf);
+	}
+	check_plan();
+}
+
+int
+main(void)
+{
+	plan(2);
+	datetime_test();
+	tostring_datetime_test();
+
+	return check_plan();
+}
diff --git a/test/unit/datetime.result b/test/unit/datetime.result
new file mode 100644
index 000000000..33997d9df
--- /dev/null
+++ b/test/unit/datetime.result
@@ -0,0 +1,358 @@
+1..1
+    1..355
+    ok 1 - correct parse_datetime return value for '2012-12-24 15:30Z'
+    ok 2 - correct parse_datetime output seconds for '2012-12-24 15:30Z
+    ok 3 - strftime
+    ok 4 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 5 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 6 - correct parse_datetime return value for '2012-12-24 15:30z'
+    ok 7 - correct parse_datetime output seconds for '2012-12-24 15:30z
+    ok 8 - strftime
+    ok 9 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 10 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 11 - correct parse_datetime return value for '2012-12-24 15:30'
+    ok 12 - correct parse_datetime output seconds for '2012-12-24 15:30
+    ok 13 - strftime
+    ok 14 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 15 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 16 - correct parse_datetime return value for '2012-12-24 16:30+01:00'
+    ok 17 - correct parse_datetime output seconds for '2012-12-24 16:30+01:00
+    ok 18 - strftime
+    ok 19 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 20 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 21 - correct parse_datetime return value for '2012-12-24 16:30+0100'
+    ok 22 - correct parse_datetime output seconds for '2012-12-24 16:30+0100
+    ok 23 - strftime
+    ok 24 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 25 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 26 - correct parse_datetime return value for '2012-12-24 16:30+01'
+    ok 27 - correct parse_datetime output seconds for '2012-12-24 16:30+01
+    ok 28 - strftime
+    ok 29 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 30 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 31 - correct parse_datetime return value for '2012-12-24 14:30-01:00'
+    ok 32 - correct parse_datetime output seconds for '2012-12-24 14:30-01:00
+    ok 33 - strftime
+    ok 34 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 35 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 36 - correct parse_datetime return value for '2012-12-24 14:30-0100'
+    ok 37 - correct parse_datetime output seconds for '2012-12-24 14:30-0100
+    ok 38 - strftime
+    ok 39 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 40 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 41 - correct parse_datetime return value for '2012-12-24 14:30-01'
+    ok 42 - correct parse_datetime output seconds for '2012-12-24 14:30-01
+    ok 43 - strftime
+    ok 44 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 45 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 46 - correct parse_datetime return value for '2012-12-24 15:30:00Z'
+    ok 47 - correct parse_datetime output seconds for '2012-12-24 15:30:00Z
+    ok 48 - strftime
+    ok 49 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 50 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 51 - correct parse_datetime return value for '2012-12-24 15:30:00z'
+    ok 52 - correct parse_datetime output seconds for '2012-12-24 15:30:00z
+    ok 53 - strftime
+    ok 54 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 55 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 56 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 57 - correct parse_datetime output seconds for '2012-12-24 15:30:00
+    ok 58 - strftime
+    ok 59 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 60 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 61 - correct parse_datetime return value for '2012-12-24 16:30:00+01:00'
+    ok 62 - correct parse_datetime output seconds for '2012-12-24 16:30:00+01:00
+    ok 63 - strftime
+    ok 64 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 65 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 66 - correct parse_datetime return value for '2012-12-24 16:30:00+0100'
+    ok 67 - correct parse_datetime output seconds for '2012-12-24 16:30:00+0100
+    ok 68 - strftime
+    ok 69 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 70 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 71 - correct parse_datetime return value for '2012-12-24 14:30:00-01:00'
+    ok 72 - correct parse_datetime output seconds for '2012-12-24 14:30:00-01:00
+    ok 73 - strftime
+    ok 74 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 75 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 76 - correct parse_datetime return value for '2012-12-24 14:30:00-0100'
+    ok 77 - correct parse_datetime output seconds for '2012-12-24 14:30:00-0100
+    ok 78 - strftime
+    ok 79 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 80 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 81 - correct parse_datetime return value for '2012-12-24 15:30:00.123456Z'
+    ok 82 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456Z
+    ok 83 - strftime
+    ok 84 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 85 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 86 - correct parse_datetime return value for '2012-12-24 15:30:00.123456z'
+    ok 87 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456z
+    ok 88 - strftime
+    ok 89 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 90 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 91 - correct parse_datetime return value for '2012-12-24 15:30:00.123456'
+    ok 92 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456
+    ok 93 - strftime
+    ok 94 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 95 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 96 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01:00'
+    ok 97 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01:00
+    ok 98 - strftime
+    ok 99 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 100 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 101 - correct parse_datetime return value for '2012-12-24 16:30:00.123456+01'
+    ok 102 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456+01
+    ok 103 - strftime
+    ok 104 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 105 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 106 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01:00'
+    ok 107 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01:00
+    ok 108 - strftime
+    ok 109 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 110 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 111 - correct parse_datetime return value for '2012-12-24 14:30:00.123456-01'
+    ok 112 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456-01
+    ok 113 - strftime
+    ok 114 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 115 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 116 - correct parse_datetime return value for '2012-12-24t15:30Z'
+    ok 117 - correct parse_datetime output seconds for '2012-12-24t15:30Z
+    ok 118 - strftime
+    ok 119 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 120 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 121 - correct parse_datetime return value for '2012-12-24t15:30z'
+    ok 122 - correct parse_datetime output seconds for '2012-12-24t15:30z
+    ok 123 - strftime
+    ok 124 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 125 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 126 - correct parse_datetime return value for '2012-12-24t15:30'
+    ok 127 - correct parse_datetime output seconds for '2012-12-24t15:30
+    ok 128 - strftime
+    ok 129 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 130 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 131 - correct parse_datetime return value for '2012-12-24t16:30+01:00'
+    ok 132 - correct parse_datetime output seconds for '2012-12-24t16:30+01:00
+    ok 133 - strftime
+    ok 134 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 135 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 136 - correct parse_datetime return value for '2012-12-24t16:30+0100'
+    ok 137 - correct parse_datetime output seconds for '2012-12-24t16:30+0100
+    ok 138 - strftime
+    ok 139 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 140 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 141 - correct parse_datetime return value for '2012-12-24t14:30-01:00'
+    ok 142 - correct parse_datetime output seconds for '2012-12-24t14:30-01:00
+    ok 143 - strftime
+    ok 144 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 145 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 146 - correct parse_datetime return value for '2012-12-24t14:30-0100'
+    ok 147 - correct parse_datetime output seconds for '2012-12-24t14:30-0100
+    ok 148 - strftime
+    ok 149 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 150 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 151 - correct parse_datetime return value for '2012-12-24t15:30:00Z'
+    ok 152 - correct parse_datetime output seconds for '2012-12-24t15:30:00Z
+    ok 153 - strftime
+    ok 154 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 155 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 156 - correct parse_datetime return value for '2012-12-24t15:30:00z'
+    ok 157 - correct parse_datetime output seconds for '2012-12-24t15:30:00z
+    ok 158 - strftime
+    ok 159 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 160 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 161 - correct parse_datetime return value for '2012-12-24t15:30:00'
+    ok 162 - correct parse_datetime output seconds for '2012-12-24t15:30:00
+    ok 163 - strftime
+    ok 164 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 165 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 166 - correct parse_datetime return value for '2012-12-24t16:30:00+01:00'
+    ok 167 - correct parse_datetime output seconds for '2012-12-24t16:30:00+01:00
+    ok 168 - strftime
+    ok 169 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 170 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 171 - correct parse_datetime return value for '2012-12-24t16:30:00+0100'
+    ok 172 - correct parse_datetime output seconds for '2012-12-24t16:30:00+0100
+    ok 173 - strftime
+    ok 174 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 175 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 176 - correct parse_datetime return value for '2012-12-24t14:30:00-01:00'
+    ok 177 - correct parse_datetime output seconds for '2012-12-24t14:30:00-01:00
+    ok 178 - strftime
+    ok 179 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 180 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 181 - correct parse_datetime return value for '2012-12-24t14:30:00-0100'
+    ok 182 - correct parse_datetime output seconds for '2012-12-24t14:30:00-0100
+    ok 183 - strftime
+    ok 184 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 185 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 186 - correct parse_datetime return value for '2012-12-24t15:30:00.123456Z'
+    ok 187 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456Z
+    ok 188 - strftime
+    ok 189 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 190 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 191 - correct parse_datetime return value for '2012-12-24t15:30:00.123456z'
+    ok 192 - correct parse_datetime output seconds for '2012-12-24t15:30:00.123456z
+    ok 193 - strftime
+    ok 194 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 195 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 196 - correct parse_datetime return value for '2012-12-24t16:30:00.123456+01:00'
+    ok 197 - correct parse_datetime output seconds for '2012-12-24t16:30:00.123456+01:00
+    ok 198 - strftime
+    ok 199 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 200 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 201 - correct parse_datetime return value for '2012-12-24t14:30:00.123456-01:00'
+    ok 202 - correct parse_datetime output seconds for '2012-12-24t14:30:00.123456-01:00
+    ok 203 - strftime
+    ok 204 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 205 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 206 - correct parse_datetime return value for '2012-12-24 16:30 +01:00'
+    ok 207 - correct parse_datetime output seconds for '2012-12-24 16:30 +01:00
+    ok 208 - strftime
+    ok 209 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 210 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 211 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
+    ok 212 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
+    ok 213 - strftime
+    ok 214 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 215 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 216 - correct parse_datetime return value for '2012-12-24 15:30 UTC'
+    ok 217 - correct parse_datetime output seconds for '2012-12-24 15:30 UTC
+    ok 218 - strftime
+    ok 219 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 220 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 221 - correct parse_datetime return value for '2012-12-24 16:30 UTC+1'
+    ok 222 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+1
+    ok 223 - strftime
+    ok 224 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 225 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 226 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01'
+    ok 227 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01
+    ok 228 - strftime
+    ok 229 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 230 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 231 - correct parse_datetime return value for '2012-12-24 16:30 UTC+0100'
+    ok 232 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+0100
+    ok 233 - strftime
+    ok 234 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 235 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 236 - correct parse_datetime return value for '2012-12-24 16:30 UTC+01:00'
+    ok 237 - correct parse_datetime output seconds for '2012-12-24 16:30 UTC+01:00
+    ok 238 - strftime
+    ok 239 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 240 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 241 - correct parse_datetime return value for '2012-12-24 14:30 UTC-1'
+    ok 242 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-1
+    ok 243 - strftime
+    ok 244 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 245 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 246 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01'
+    ok 247 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01
+    ok 248 - strftime
+    ok 249 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 250 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 251 - correct parse_datetime return value for '2012-12-24 14:30 UTC-01:00'
+    ok 252 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-01:00
+    ok 253 - strftime
+    ok 254 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 255 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 256 - correct parse_datetime return value for '2012-12-24 14:30 UTC-0100'
+    ok 257 - correct parse_datetime output seconds for '2012-12-24 14:30 UTC-0100
+    ok 258 - strftime
+    ok 259 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 260 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 261 - correct parse_datetime return value for '2012-12-24 15:30 GMT'
+    ok 262 - correct parse_datetime output seconds for '2012-12-24 15:30 GMT
+    ok 263 - strftime
+    ok 264 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 265 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 266 - correct parse_datetime return value for '2012-12-24 16:30 GMT+1'
+    ok 267 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+1
+    ok 268 - strftime
+    ok 269 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 270 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 271 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01'
+    ok 272 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01
+    ok 273 - strftime
+    ok 274 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 275 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 276 - correct parse_datetime return value for '2012-12-24 16:30 GMT+0100'
+    ok 277 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+0100
+    ok 278 - strftime
+    ok 279 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 280 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 281 - correct parse_datetime return value for '2012-12-24 16:30 GMT+01:00'
+    ok 282 - correct parse_datetime output seconds for '2012-12-24 16:30 GMT+01:00
+    ok 283 - strftime
+    ok 284 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 285 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 286 - correct parse_datetime return value for '2012-12-24 14:30 GMT-1'
+    ok 287 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-1
+    ok 288 - strftime
+    ok 289 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 290 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 291 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01'
+    ok 292 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01
+    ok 293 - strftime
+    ok 294 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 295 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 296 - correct parse_datetime return value for '2012-12-24 14:30 GMT-01:00'
+    ok 297 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-01:00
+    ok 298 - strftime
+    ok 299 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 300 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 301 - correct parse_datetime return value for '2012-12-24 14:30 GMT-0100'
+    ok 302 - correct parse_datetime output seconds for '2012-12-24 14:30 GMT-0100
+    ok 303 - strftime
+    ok 304 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 305 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 306 - correct parse_datetime return value for '2012-12-24 14:30 -01:00'
+    ok 307 - correct parse_datetime output seconds for '2012-12-24 14:30 -01:00
+    ok 308 - strftime
+    ok 309 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 310 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 311 - correct parse_datetime return value for '2012-12-24 16:30:00 +01:00'
+    ok 312 - correct parse_datetime output seconds for '2012-12-24 16:30:00 +01:00
+    ok 313 - strftime
+    ok 314 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 315 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 316 - correct parse_datetime return value for '2012-12-24 14:30:00 -01:00'
+    ok 317 - correct parse_datetime output seconds for '2012-12-24 14:30:00 -01:00
+    ok 318 - strftime
+    ok 319 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 320 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 321 - correct parse_datetime return value for '2012-12-24 16:30:00.123456 +01:00'
+    ok 322 - correct parse_datetime output seconds for '2012-12-24 16:30:00.123456 +01:00
+    ok 323 - strftime
+    ok 324 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 325 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 326 - correct parse_datetime return value for '2012-12-24 14:30:00.123456 -01:00'
+    ok 327 - correct parse_datetime output seconds for '2012-12-24 14:30:00.123456 -01:00
+    ok 328 - strftime
+    ok 329 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 330 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 331 - correct parse_datetime return value for '2012-12-24 15:30:00.123456 -00:00'
+    ok 332 - correct parse_datetime output seconds for '2012-12-24 15:30:00.123456 -00:00
+    ok 333 - strftime
+    ok 334 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 335 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 336 - correct parse_datetime return value for '20121224T1630+01:00'
+    ok 337 - correct parse_datetime output seconds for '20121224T1630+01:00
+    ok 338 - strftime
+    ok 339 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 340 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 341 - correct parse_datetime return value for '2012-12-24T1630+01:00'
+    ok 342 - correct parse_datetime output seconds for '2012-12-24T1630+01:00
+    ok 343 - strftime
+    ok 344 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 345 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 346 - correct parse_datetime return value for '20121224T16:30+01'
+    ok 347 - correct parse_datetime output seconds for '20121224T16:30+01
+    ok 348 - strftime
+    ok 349 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 350 - reversible seconds via strftime for '2012-12-24 15:30:00
+    ok 351 - correct parse_datetime return value for '20121224T16:30 +01'
+    ok 352 - correct parse_datetime output seconds for '20121224T16:30 +01
+    ok 353 - strftime
+    ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00'
+    ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00
+ok 1 - subtests
diff --git a/third_party/c-dt b/third_party/c-dt
new file mode 160000
index 000000000..3cbbbc7f0
--- /dev/null
+++ b/third_party/c-dt
@@ -0,0 +1 @@
+Subproject commit 3cbbbc7f032cfa67a8df9f81101403249825d7f3
-- 
2.29.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [Tarantool-patches] [PATCH v6 2/5] box, datetime: messagepack support for datetime
  2021-08-19  2:56 [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation Timur Safin via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime Timur Safin via Tarantool-patches
@ 2021-08-19  2:56 ` Timur Safin via Tarantool-patches
  2021-08-19  9:58   ` Serge Petrenko via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 20+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-19  2:56 UTC (permalink / raw)
  To: vdavydov, sergepetrenko, tarantool-patches; +Cc: v.shpilevoy

Serialize `struct datetime` as newly introduced MP_EXT type.
It saves 1 required integer field and upto 2 optional
unsigned fields in very compact fashion.
- secs is required field;
- but nsec, offset are both optional;

* json, yaml serialization formats, lua output mode
  supported;
* exported symbols for datetime messagepack size calculations
  so they are available for usage on Lua side.

* As a bonus we introduce core/mp_utils.h with set of helpers
  which simplify working with mp_sizeof*/mp_encode_*/mp_decode_*
  functions regardless of a signedness of an integer data we
  deal with.

Part of #5941
Part of #5946
---
 extra/exports                     |   3 +
 src/box/lua/serialize_lua.c       |   8 +-
 src/box/msgpack.c                 |   7 +-
 src/lib/core/CMakeLists.txt       |   4 +-
 src/lib/core/datetime.h           |  24 +++++
 src/lib/core/mp_datetime.c        | 171 ++++++++++++++++++++++++++++++
 src/lib/core/mp_datetime.h        |  64 +++++++++++
 src/lib/core/mp_extension_types.h |   1 +
 src/lib/core/mp_utils.h           |  64 +++++++++++
 src/lib/mpstream/mpstream.c       |  11 ++
 src/lib/mpstream/mpstream.h       |   4 +
 src/lua/msgpack.c                 |  12 +++
 src/lua/msgpackffi.lua            |  18 ++++
 src/lua/serializer.c              |   4 +
 src/lua/serializer.h              |   2 +
 test/unit/datetime.c              | 124 +++++++++++++++++++++-
 test/unit/datetime.result         | 115 +++++++++++++++++++-
 third_party/lua-cjson/lua_cjson.c |   8 ++
 third_party/lua-yaml/lyaml.cc     |   6 +-
 19 files changed, 644 insertions(+), 6 deletions(-)
 create mode 100644 src/lib/core/mp_datetime.c
 create mode 100644 src/lib/core/mp_datetime.h
 create mode 100644 src/lib/core/mp_utils.h

diff --git a/extra/exports b/extra/exports
index 0e7392d61..f232c94e0 100644
--- a/extra/exports
+++ b/extra/exports
@@ -149,6 +149,7 @@ csv_iterator_create
 csv_next
 csv_setopt
 datetime_now
+datetime_pack
 datetime_strftime
 datetime_to_string
 datetime_unpack
@@ -378,6 +379,7 @@ mp_decode_uint
 mp_encode_array
 mp_encode_bin
 mp_encode_bool
+mp_encode_datetime
 mp_encode_decimal
 mp_encode_double
 mp_encode_float
@@ -394,6 +396,7 @@ mp_next
 mp_next_slowpath
 mp_parser_hint
 mp_sizeof_array
+mp_sizeof_datetime
 mp_sizeof_decimal
 mp_sizeof_str
 mp_sizeof_uuid
diff --git a/src/box/lua/serialize_lua.c b/src/box/lua/serialize_lua.c
index 1f791980f..2c1ed6abb 100644
--- a/src/box/lua/serialize_lua.c
+++ b/src/box/lua/serialize_lua.c
@@ -768,7 +768,7 @@ static int
 dump_node(struct lua_dumper *d, struct node *nd, int indent)
 {
 	struct luaL_field *field = &nd->field;
-	char buf[FPCONV_G_FMT_BUFSIZE];
+	char buf[MAX(FPCONV_G_FMT_BUFSIZE, DT_TO_STRING_BUFSIZE)];
 	int ltype = lua_type(d->L, -1);
 	const char *str = NULL;
 	size_t len = 0;
@@ -861,6 +861,12 @@ dump_node(struct lua_dumper *d, struct node *nd, int indent)
 			str = tt_uuid_str(field->uuidval);
 			len = UUID_STR_LEN;
 			break;
+		case MP_DATETIME:
+			nd->mask |= NODE_QUOTE;
+			str = buf;
+			len = datetime_to_string(field->dateval, buf,
+						 sizeof(buf));
+			break;
 		default:
 			d->err = EINVAL;
 			snprintf(d->err_msg, sizeof(d->err_msg),
diff --git a/src/box/msgpack.c b/src/box/msgpack.c
index 1723dea4c..12f4fd95a 100644
--- a/src/box/msgpack.c
+++ b/src/box/msgpack.c
@@ -1,5 +1,5 @@
 /*
- * Copyright 2020, Tarantool AUTHORS, please see AUTHORS file.
+ * Copyright 2020-2021, Tarantool AUTHORS, please see AUTHORS file.
  *
  * Redistribution and use in source and binary forms, with or
  * without modification, are permitted provided that the following
@@ -35,6 +35,7 @@
 #include "mp_decimal.h"
 #include "uuid/mp_uuid.h"
 #include "mp_error.h"
+#include "mp_datetime.h"
 
 static int
 msgpack_fprint_ext(FILE *file, const char **data, int depth)
@@ -47,6 +48,8 @@ msgpack_fprint_ext(FILE *file, const char **data, int depth)
 		return mp_fprint_decimal(file, data, len);
 	case MP_UUID:
 		return mp_fprint_uuid(file, data, len);
+	case MP_DATETIME:
+		return mp_fprint_datetime(file, data, len);
 	case MP_ERROR:
 		return mp_fprint_error(file, data, depth);
 	default:
@@ -65,6 +68,8 @@ msgpack_snprint_ext(char *buf, int size, const char **data, int depth)
 		return mp_snprint_decimal(buf, size, data, len);
 	case MP_UUID:
 		return mp_snprint_uuid(buf, size, data, len);
+	case MP_DATETIME:
+		return mp_snprint_datetime(buf, size, data, len);
 	case MP_ERROR:
 		return mp_snprint_error(buf, size, data, depth);
 	default:
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index 8bc776b82..61fc6548f 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -31,6 +31,7 @@ set(core_sources
     mp_decimal.c
     cord_buf.c
     datetime.c
+    mp_datetime.c
 )
 
 if (TARGET_OS_NETBSD)
@@ -44,7 +45,8 @@ add_library(core STATIC ${core_sources})
 
 target_link_libraries(core salad small uri decNumber bit ${LIBEV_LIBRARIES}
                       ${LIBEIO_LIBRARIES} ${LIBCORO_LIBRARIES}
-                      ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES})
+                      ${MSGPUCK_LIBRARIES} ${ICU_LIBRARIES}
+                      ${LIBCDT_LIBRARIES})
 
 if (ENABLE_BACKTRACE AND NOT TARGET_OS_DARWIN)
     target_link_libraries(core gcc_s ${UNWIND_LIBRARIES})
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
index 71feefded..fb537e372 100644
--- a/src/lib/core/datetime.h
+++ b/src/lib/core/datetime.h
@@ -5,6 +5,7 @@
  * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
  */
 
+#include <limits.h>
 #include <stdint.h>
 #include <stdbool.h>
 #include "c-dt/dt.h"
@@ -30,8 +31,26 @@ extern "C"
 #define DT_EPOCH_1970_OFFSET  719163
 #endif
 
+/**
+ * c-dt library uses int as type for dt value, which
+ * represents the number of days since Rata Die date.
+ * This implies limits to the number of seconds we
+ * could safely store in our structures and then safely
+ * pass to c-dt functions.
+ *
+ * So supported ranges will be
+ * - for seconds [-185604722870400 .. 185480451417600]
+ * - for dates   [-5879610-06-22T00:00Z .. 5879611-07-11T00:00Z]
+ */
+#define MAX_DT_DAY_VALUE (int64_t)INT_MAX
+#define MIN_DT_DAY_VALUE (int64_t)INT_MIN
 #define SECS_EPOCH_1970_OFFSET 	\
 	((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
+#define MAX_EPOCH_SECS_VALUE    \
+	(MAX_DT_DAY_VALUE * SECS_PER_DAY - SECS_EPOCH_1970_OFFSET)
+#define MIN_EPOCH_SECS_VALUE    \
+	(MIN_DT_DAY_VALUE * SECS_PER_DAY - SECS_EPOCH_1970_OFFSET)
+
 /**
  * datetime structure keeps number of seconds since
  * Unix Epoch.
@@ -57,6 +76,11 @@ struct datetime_interval {
 	uint32_t nsec;
 };
 
+/**
+ * Required size of datetime_to_string string buffer
+ */
+#define DT_TO_STRING_BUFSIZE   48
+
 /**
  * Convert datetime to string using default format
  * @param date source datetime value
diff --git a/src/lib/core/mp_datetime.c b/src/lib/core/mp_datetime.c
new file mode 100644
index 000000000..1430517e4
--- /dev/null
+++ b/src/lib/core/mp_datetime.c
@@ -0,0 +1,171 @@
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#include <limits.h>
+
+#include "msgpuck.h"
+#include "mp_datetime.h"
+#include "mp_extension_types.h"
+#include "mp_utils.h"
+
+/*
+  Datetime MessagePack serialization schema is MP_EXT (0xC7 for 1 byte length)
+  extension, which creates container of 1 to 3 integers.
+
+  +----+-----------+---+====~~~~~~~====+-----~~~~~~~~-------+....~~~~~~~....+
+  |0xC7|len (uint8)| 4 | seconds (int) | nanoseconds (uint) | offset (int)  |
+  +----+-----------+---+====~~~~~~~====+-----~~~~~~~~-------+....~~~~~~~....+
+
+  MessagePack extension MP_EXT (0xC7), after 1-byte length, contains:
+
+  - signed integer seconds part (required). Depending on the value of
+    seconds it may be from 1 to 8 bytes positive or negative integer number;
+
+  - [optional] fraction time in nanoseconds as unsigned integer.
+    If this value is 0 then it's not saved (unless there is offset field,
+    as below);
+
+  - [optional] timezone offset in minutes as signed integer.
+    If this field is 0 then it's not saved.
+ */
+
+
+#define check_secs(secs)                                \
+	assert((int64_t)(secs) <= MAX_EPOCH_SECS_VALUE);\
+	assert((int64_t)(secs) >= MIN_EPOCH_SECS_VALUE);
+
+#define check_nanosecs(nsec)      assert((nsec) < 1000000000);
+
+#define check_tz_offset(offset)       \
+	assert((offset) <= (12 * 60));\
+	assert((offset) >= (-12 * 60));
+
+static inline uint32_t
+mp_sizeof_datetime_raw(const struct datetime *date)
+{
+	check_secs(date->secs);
+	uint32_t sz = mp_sizeof_xint(date->secs);
+
+	/*
+	 * even if nanosecs == 0 we need to output something
+	 * if we have a non-null tz offset
+	 */
+	if (date->nsec != 0 || date->offset != 0) {
+		check_nanosecs(date->nsec);
+		sz += mp_sizeof_xint(date->nsec);
+	}
+	if (date->offset != 0) {
+		check_tz_offset(date->offset);
+		sz += mp_sizeof_xint(date->offset);
+	}
+	return sz;
+}
+
+uint32_t
+mp_sizeof_datetime(const struct datetime *date)
+{
+	return mp_sizeof_ext(mp_sizeof_datetime_raw(date));
+}
+
+struct datetime *
+datetime_unpack(const char **data, uint32_t len, struct datetime *date)
+{
+	const char *svp = *data;
+
+	memset(date, 0, sizeof(*date));
+
+	int64_t seconds = mp_decode_xint(data);
+	check_secs(seconds);
+	date->secs = seconds;
+
+	len -= *data - svp;
+	if (len <= 0)
+		return date;
+
+	svp = *data;
+	uint64_t nanoseconds = mp_decode_uint(data);
+	check_nanosecs(nanoseconds);
+	date->nsec = nanoseconds;
+	len -= *data - svp;
+
+	if (len <= 0)
+		return date;
+
+	int64_t offset = mp_decode_xint(data);
+	check_tz_offset(offset);
+	date->offset = offset;
+
+	return date;
+}
+
+struct datetime *
+mp_decode_datetime(const char **data, struct datetime *date)
+{
+	if (mp_typeof(**data) != MP_EXT)
+		return NULL;
+
+	const char *svp = *data;
+	int8_t type;
+	uint32_t len = mp_decode_extl(data, &type);
+
+	if (type != MP_DATETIME || len == 0) {
+		*data = svp;
+		return NULL;
+	}
+	return datetime_unpack(data, len, date);
+}
+
+char *
+datetime_pack(char *data, const struct datetime *date)
+{
+	data = mp_encode_xint(data, date->secs);
+	if (date->nsec != 0 || date->offset != 0)
+		data = mp_encode_uint(data, date->nsec);
+	if (date->offset)
+		data = mp_encode_xint(data, date->offset);
+
+	return data;
+}
+
+char *
+mp_encode_datetime(char *data, const struct datetime *date)
+{
+	uint32_t len = mp_sizeof_datetime_raw(date);
+
+	data = mp_encode_extl(data, MP_DATETIME, len);
+
+	return datetime_pack(data, date);
+}
+
+int
+mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len)
+{
+	struct datetime date = {
+		.secs = 0, .nsec = 0, .offset = 0
+	};
+
+	if (datetime_unpack(data, len, &date) == NULL)
+		return -1;
+
+	return datetime_to_string(&date, buf, size);
+}
+
+int
+mp_fprint_datetime(FILE *file, const char **data, uint32_t len)
+{
+	struct datetime date = {
+		.secs = 0, .nsec = 0, .offset = 0
+	};
+
+	if (datetime_unpack(data, len, &date) == NULL)
+		return -1;
+
+	char buf[DT_TO_STRING_BUFSIZE];
+	datetime_to_string(&date, buf, sizeof(buf));
+
+	return fprintf(file, "%s", buf);
+}
+
diff --git a/src/lib/core/mp_datetime.h b/src/lib/core/mp_datetime.h
new file mode 100644
index 000000000..92e94a243
--- /dev/null
+++ b/src/lib/core/mp_datetime.h
@@ -0,0 +1,64 @@
+#pragma once
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#include <stdio.h>
+#include "datetime.h"
+
+#if defined(__cplusplus)
+extern "C"
+{
+#endif /* defined(__cplusplus) */
+
+/**
+ * Unpack datetime data from MessagePack buffer.
+ * @sa datetime_pack
+ */
+struct datetime *
+datetime_unpack(const char **data, uint32_t len, struct datetime *date);
+
+/**
+ * Pack datetime data to MessagePack buffer.
+ * @sa datetime_unpack
+ */
+char *
+datetime_pack(char *data, const struct datetime *date);
+
+/**
+ * Calculate size of MessagePack buffer for datetime data.
+ */
+uint32_t
+mp_sizeof_datetime(const struct datetime *date);
+
+/**
+ * Decode data from MessagePack buffer to datetime structure.
+ */
+struct datetime *
+mp_decode_datetime(const char **data, struct datetime *date);
+
+/**
+ * Encode datetime structure to the MessagePack buffer.
+ */
+char *
+mp_encode_datetime(char *data, const struct datetime *date);
+
+/**
+ * Print datetime's string representation into a given buffer.
+ * @sa mp_snprint_decimal
+ */
+int
+mp_snprint_datetime(char *buf, int size, const char **data, uint32_t len);
+
+/**
+ * Print datetime's string representation into a stream.
+ * @sa mp_fprint_decimal
+ */
+int
+mp_fprint_datetime(FILE *file, const char **data, uint32_t len);
+
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
diff --git a/src/lib/core/mp_extension_types.h b/src/lib/core/mp_extension_types.h
index e3ff9f5d0..3b7eaee7c 100644
--- a/src/lib/core/mp_extension_types.h
+++ b/src/lib/core/mp_extension_types.h
@@ -44,6 +44,7 @@ enum mp_extension_type {
     MP_DECIMAL = 1,
     MP_UUID = 2,
     MP_ERROR = 3,
+    MP_DATETIME = 4,
     mp_extension_type_MAX,
 };
 
diff --git a/src/lib/core/mp_utils.h b/src/lib/core/mp_utils.h
new file mode 100644
index 000000000..bcfecca2a
--- /dev/null
+++ b/src/lib/core/mp_utils.h
@@ -0,0 +1,64 @@
+#pragma once
+/*
+ * SPDX-License-Identifier: BSD-2-Clause
+ *
+ * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
+ */
+
+#include <limits.h>
+#include <assert.h>
+
+#include "msgpuck.h"
+
+/**
+ * Convenient wrapper for determining size of integer value
+ * in MessagePack, regardless of it's sign.
+ *
+ * @param n signed 64-bit value
+ * @sa mp_sizeof_int
+ * @sa mp_sizeof_uint
+ */
+static inline uint32_t
+mp_sizeof_xint(int64_t n)
+{
+	return n < 0 ? mp_sizeof_int(n) : mp_sizeof_uint(n);
+}
+
+/**
+ * Convenient wrapper for encoding integer value
+ * to messagepack, regardless of it's sign.
+ *
+ * @param data output MessagePack buffer
+ * @param v 64-bit value to be encoded.
+ * @sa mp_encode_int
+ * @sa mp_encode_uint
+ */
+static inline char *
+mp_encode_xint(char *data, int64_t v)
+{
+	assert(v < 0 || (uint64_t)v <= LONG_MAX);
+	return v < 0 ? mp_encode_int(data, v) : mp_encode_uint(data, v);
+}
+
+/**
+ * Convenient wrapper for decoding to integer value
+ * from MessagePack, regardless of it's sign.
+ *
+ * @param data messagepack buffer
+ * @retval return signed, 64-bit value.
+ * @sa mp_decode_int
+ * @sa mp_decode_uint
+ */
+static inline int64_t
+mp_decode_xint(const char **data)
+{
+	switch (mp_typeof(**data)) {
+	case MP_UINT:
+		return (int64_t)mp_decode_uint(data);
+	case MP_INT:
+		return mp_decode_int(data);
+	default:
+		mp_unreachable();
+	}
+	return 0;
+}
diff --git a/src/lib/mpstream/mpstream.c b/src/lib/mpstream/mpstream.c
index 70ca29889..d3e1de965 100644
--- a/src/lib/mpstream/mpstream.c
+++ b/src/lib/mpstream/mpstream.c
@@ -35,6 +35,7 @@
 #include "msgpuck.h"
 #include "mp_decimal.h"
 #include "uuid/mp_uuid.h"
+#include "mp_datetime.h"
 
 void
 mpstream_reserve_slow(struct mpstream *stream, size_t size)
@@ -208,6 +209,16 @@ mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid)
 	mpstream_advance(stream, pos - data);
 }
 
+void
+mpstream_encode_datetime(struct mpstream *stream, const struct datetime *val)
+{
+	char *data = mpstream_reserve(stream, mp_sizeof_datetime(val));
+	if (data == NULL)
+		return;
+	char *pos = mp_encode_datetime(data, val);
+	mpstream_advance(stream, pos - data);
+}
+
 void
 mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n)
 {
diff --git a/src/lib/mpstream/mpstream.h b/src/lib/mpstream/mpstream.h
index a60add143..94831160f 100644
--- a/src/lib/mpstream/mpstream.h
+++ b/src/lib/mpstream/mpstream.h
@@ -39,6 +39,7 @@ extern "C" {
 #endif /* defined(__cplusplus) */
 
 struct tt_uuid;
+struct datetime;
 
 /**
 * Ask the allocator to reserve at least size bytes. It can reserve
@@ -145,6 +146,9 @@ mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val);
 void
 mpstream_encode_uuid(struct mpstream *stream, const struct tt_uuid *uuid);
 
+void
+mpstream_encode_datetime(struct mpstream *stream, const struct datetime *dt);
+
 /** Copies n bytes from memory area src to stream. */
 void
 mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n);
diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
index b6ecf2b1e..bca53f6b7 100644
--- a/src/lua/msgpack.c
+++ b/src/lua/msgpack.c
@@ -46,6 +46,7 @@
 #include "lib/core/decimal.h" /* decimal_unpack() */
 #include "lib/uuid/mp_uuid.h" /* mp_decode_uuid() */
 #include "lib/core/mp_extension_types.h"
+#include "lib/core/mp_datetime.h"
 
 #include "cord_buf.h"
 #include <fiber.h>
@@ -200,6 +201,9 @@ restart: /* used by MP_EXT of unidentified subtype */
 			break;
 		case MP_ERROR:
 			return luamp_encode_extension(L, top, stream);
+		case MP_DATETIME:
+			mpstream_encode_datetime(stream, field->dateval);
+			break;
 		default:
 			/* Run trigger if type can't be encoded */
 			type = luamp_encode_extension(L, top, stream);
@@ -333,6 +337,14 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
 				goto ext_decode_err;
 			return;
 		}
+		case MP_DATETIME:
+		{
+			struct datetime *date = luaL_pushdatetime(L);
+			date = datetime_unpack(data, len, date);
+			if (date == NULL)
+				goto ext_decode_err;
+			return;
+		}
 		default:
 			/* reset data to the extension header */
 			*data = svp;
diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua
index 1d54f11b8..fb5e7d644 100644
--- a/src/lua/msgpackffi.lua
+++ b/src/lua/msgpackffi.lua
@@ -26,6 +26,10 @@ char *
 mp_encode_uuid(char *data, const struct tt_uuid *uuid);
 uint32_t
 mp_sizeof_uuid();
+uint32_t
+mp_sizeof_datetime(const struct t_datetime_tz *date);
+char *
+mp_encode_datetime(char *data, const struct t_datetime_tz *date);
 float
 mp_decode_float(const char **data);
 double
@@ -36,6 +40,8 @@ decimal_t *
 decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
 struct tt_uuid *
 uuid_unpack(const char **data, uint32_t len, struct tt_uuid *uuid);
+struct datetime *
+datetime_unpack(const char **data, uint32_t len, struct datetime *date);
 ]])
 
 local strict_alignment = (jit.arch == 'arm')
@@ -142,6 +148,11 @@ local function encode_uuid(buf, uuid)
     builtin.mp_encode_uuid(p, uuid)
 end
 
+local function encode_datetime(buf, date)
+    local p = buf:alloc(builtin.mp_sizeof_datetime(date))
+    builtin.mp_encode_datetime(p, date)
+end
+
 local function encode_int(buf, num)
     if num >= 0 then
         if num <= 0x7f then
@@ -320,6 +331,7 @@ on_encode(ffi.typeof('float'), encode_float)
 on_encode(ffi.typeof('double'), encode_double)
 on_encode(ffi.typeof('decimal_t'), encode_decimal)
 on_encode(ffi.typeof('struct tt_uuid'), encode_uuid)
+on_encode(ffi.typeof('struct datetime'), encode_datetime)
 
 --------------------------------------------------------------------------------
 -- Decoder
@@ -513,6 +525,12 @@ local ext_decoder = {
         builtin.uuid_unpack(data, len, uuid)
         return uuid
     end,
+    -- MP_DATETIME
+    [4] = function(data, len)
+        local dt = ffi.new("struct datetime")
+        builtin.datetime_unpack(data, len, dt)
+        return dt
+    end,
 }
 
 local function decode_ext(data)
diff --git a/src/lua/serializer.c b/src/lua/serializer.c
index 8db6746a3..24f4a5ff9 100644
--- a/src/lua/serializer.c
+++ b/src/lua/serializer.c
@@ -41,6 +41,7 @@
 #include "lib/core/mp_extension_types.h"
 #include "lua/error.h"
 
+#include "datetime.h"
 #include "trivia/util.h"
 #include "diag.h"
 #include "serializer_opts.h"
@@ -544,6 +545,9 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg,
 				   opts != NULL &&
 				   opts->error_marshaling_enabled) {
 				field->ext_type = MP_ERROR;
+			} else if (cd->ctypeid == CTID_DATETIME) {
+				field->ext_type = MP_DATETIME;
+				field->dateval = (struct datetime *)cdata;
 			} else {
 				field->ext_type = MP_UNKNOWN_EXTENSION;
 			}
diff --git a/src/lua/serializer.h b/src/lua/serializer.h
index 0a0501a74..e7a240e0a 100644
--- a/src/lua/serializer.h
+++ b/src/lua/serializer.h
@@ -52,6 +52,7 @@ extern "C" {
 #include <lauxlib.h>
 
 #include "trigger.h"
+#include "lib/core/datetime.h"
 #include "lib/core/decimal.h" /* decimal_t */
 #include "lib/core/mp_extension_types.h"
 #include "lua/error.h"
@@ -223,6 +224,7 @@ struct luaL_field {
 		uint32_t size;
 		decimal_t *decval;
 		struct tt_uuid *uuidval;
+		struct datetime *dateval;
 	};
 	enum mp_type type;
 	/* subtypes of MP_EXT */
diff --git a/test/unit/datetime.c b/test/unit/datetime.c
index 931636172..c6ba444f5 100644
--- a/test/unit/datetime.c
+++ b/test/unit/datetime.c
@@ -6,6 +6,9 @@
 
 #include "unit.h"
 #include "datetime.h"
+#include "mp_datetime.h"
+#include "msgpuck.h"
+#include "mp_extension_types.h"
 #include "trivia/util.h"
 
 static const char sample[] = "2012-12-24T15:30Z";
@@ -249,12 +252,131 @@ tostring_datetime_test(void)
 	check_plan();
 }
 
+static void
+mp_datetime_test()
+{
+	static struct {
+		int64_t     secs;
+		uint32_t    nsec;
+		uint32_t    offset;
+		uint32_t    len;
+	} tests[] = {
+		{/* '1970-01-01T02:00+02:00' */          0,         0,  120, 6},
+		{/* '1970-01-01T01:30+01:30' */          0,         0,   90, 6},
+		{/* '1970-01-01T01:00+01:00' */          0,         0,   60, 6},
+		{/* '1970-01-01T00:01+00:01' */          0,         0,    1, 6},
+		{/* '1970-01-01T00:00Z' */               0,         0,    0, 3},
+		{/* '1969-12-31T23:59-00:01' */          0,         0,   -1, 6},
+		{/* '1969-12-31T23:00-01:00' */          0,         0,  -60, 6},
+		{/* '1969-12-31T22:30-01:30' */          0,         0,  -90, 6},
+		{/* '1969-12-31T22:00-02:00' */          0,         0, -120, 6},
+		{/* '1970-01-01T00:00:00.123456789Z' */  0, 123456789,    0, 9},
+		{/* '1970-01-01T00:00:00.123456Z' */     0, 123456000,    0, 9},
+		{/* '1970-01-01T00:00:00.123Z' */        0, 123000000,    0, 9},
+		{/* '1973-11-29T21:33:09Z' */    123456789,         0,    0, 8},
+		{/* '2013-10-28T17:51:56Z' */   1382982716,         0,    0, 8},
+		{/* '9999-12-31T23:59:59Z' */ 253402300799,         0,    0, 12},
+		{/* '9999-12-31T23:59:59.123456789Z' */ 253402300799, 123456789, 0, 17},
+		{/* '9999-12-31T23:59:59.123456789-02:00' */ 253402300799, 123456789, -120, 18},
+	};
+	size_t index;
+
+	plan(68);
+	for (index = 0; index < lengthof(tests); index++) {
+		struct datetime date = {
+			tests[index].secs,
+			tests[index].nsec,
+			tests[index].offset
+		};
+		char buf[24], *data = buf;
+		const char *data1 = buf;
+		struct datetime ret;
+
+		char *end = mp_encode_datetime(data, &date);
+		uint32_t len = mp_sizeof_datetime(&date);
+		is(len, tests[index].len, "len %u, expected len %u",
+		   len, tests[index].len);
+		is(end - data, len,
+		   "mp_sizeof_datetime(%d) == encoded length %ld",
+		   len, end - data);
+
+		struct datetime *rc = mp_decode_datetime(&data1, &ret);
+		is(rc, &ret, "mp_decode_datetime() return code");
+		is(data1, end, "mp_sizeof_uuid() == decoded length");
+	}
+	check_plan();
+}
+
+
+static int
+mp_fprint_ext_test(FILE *file, const char **data, int depth)
+{
+	(void)depth;
+	int8_t type;
+	uint32_t len = mp_decode_extl(data, &type);
+	if (type != MP_DATETIME)
+		return fprintf(file, "undefined");
+	return mp_fprint_datetime(file, data, len);
+}
+
+static int
+mp_snprint_ext_test(char *buf, int size, const char **data, int depth)
+{
+        (void)depth;
+        int8_t type;
+        uint32_t len = mp_decode_extl(data, &type);
+        if (type != MP_DATETIME)
+                return snprintf(buf, size, "undefined");
+        return mp_snprint_datetime(buf, size, data, len);
+}
+
+static void
+mp_print_test(void)
+{
+	plan(5);
+	header();
+
+	mp_snprint_ext = mp_snprint_ext_test;
+	mp_fprint_ext = mp_fprint_ext_test;
+
+	char sample[64];
+	char buffer[64];
+	char str[64];
+	struct datetime date = {0, 0, 0}; // 1970-01-01T00:00Z
+
+	mp_encode_datetime(buffer, &date);
+	int sz = datetime_to_string(&date, str, sizeof(str));
+	int rc = mp_snprint(NULL, 0, buffer);
+	is(rc, sz, "correct mp_snprint size %u with empty buffer", rc);
+	rc = mp_snprint(str, sizeof(str), buffer);
+	is(rc, sz, "correct mp_snprint size %u", rc);
+	datetime_to_string(&date, sample, sizeof(sample));
+	is(strcmp(str, sample), 0, "correct mp_snprint result");
+
+	FILE *f = tmpfile();
+	rc = mp_fprint(f, buffer);
+	is(rc, sz, "correct mp_fprint size %u", sz);
+	rewind(f);
+	rc = fread(str, 1, sizeof(str), f);
+	str[rc] = 0;
+	is(strcmp(str, sample), 0, "correct mp_fprint result %u", rc);
+	fclose(f);
+
+	mp_snprint_ext = mp_snprint_ext_default;
+	mp_fprint_ext = mp_fprint_ext_default;
+
+	footer();
+	check_plan();
+}
+
 int
 main(void)
 {
-	plan(2);
+	plan(4);
 	datetime_test();
 	tostring_datetime_test();
+	mp_datetime_test();
+	mp_print_test();
 
 	return check_plan();
 }
diff --git a/test/unit/datetime.result b/test/unit/datetime.result
index 33997d9df..5f01c4344 100644
--- a/test/unit/datetime.result
+++ b/test/unit/datetime.result
@@ -1,4 +1,4 @@
-1..1
+1..4
     1..355
     ok 1 - correct parse_datetime return value for '2012-12-24 15:30Z'
     ok 2 - correct parse_datetime output seconds for '2012-12-24 15:30Z
@@ -356,3 +356,116 @@
     ok 354 - correct parse_datetime return value for '2012-12-24 15:30:00'
     ok 355 - reversible seconds via strftime for '2012-12-24 15:30:00
 ok 1 - subtests
+    1..15
+    ok 1 - string '1970-01-01T02:00+02:00' expected, received '1970-01-01T02:00+02:00'
+    ok 2 - string '1970-01-01T01:30+01:30' expected, received '1970-01-01T01:30+01:30'
+    ok 3 - string '1970-01-01T01:00+01:00' expected, received '1970-01-01T01:00+01:00'
+    ok 4 - string '1970-01-01T00:01+00:01' expected, received '1970-01-01T00:01+00:01'
+    ok 5 - string '1970-01-01T00:00Z' expected, received '1970-01-01T00:00Z'
+    ok 6 - string '1969-12-31T23:59-00:01' expected, received '1969-12-31T23:59-00:01'
+    ok 7 - string '1969-12-31T23:00-01:00' expected, received '1969-12-31T23:00-01:00'
+    ok 8 - string '1969-12-31T22:30-01:30' expected, received '1969-12-31T22:30-01:30'
+    ok 9 - string '1969-12-31T22:00-02:00' expected, received '1969-12-31T22:00-02:00'
+    ok 10 - string '1970-01-01T00:00:00.123456789Z' expected, received '1970-01-01T00:00:00.123456789Z'
+    ok 11 - string '1970-01-01T00:00:00.123456Z' expected, received '1970-01-01T00:00:00.123456Z'
+    ok 12 - string '1970-01-01T00:00:00.123Z' expected, received '1970-01-01T00:00:00.123Z'
+    ok 13 - string '1973-11-29T21:33:09Z' expected, received '1973-11-29T21:33:09Z'
+    ok 14 - string '2013-10-28T17:51:56Z' expected, received '2013-10-28T17:51:56Z'
+    ok 15 - string '9999-12-31T23:59:59Z' expected, received '9999-12-31T23:59:59Z'
+ok 2 - subtests
+    1..85
+    ok 1 - len 6, expected len 6
+    ok 2 - mp_sizeof_datetime(6) == encoded length 6
+    ok 3 - mp_decode_datetime() return code
+    ok 4 - mp_sizeof_uuid() == decoded length
+    ok 5 - datetime_compare(&date, &ret)
+    ok 6 - len 6, expected len 6
+    ok 7 - mp_sizeof_datetime(6) == encoded length 6
+    ok 8 - mp_decode_datetime() return code
+    ok 9 - mp_sizeof_uuid() == decoded length
+    ok 10 - datetime_compare(&date, &ret)
+    ok 11 - len 6, expected len 6
+    ok 12 - mp_sizeof_datetime(6) == encoded length 6
+    ok 13 - mp_decode_datetime() return code
+    ok 14 - mp_sizeof_uuid() == decoded length
+    ok 15 - datetime_compare(&date, &ret)
+    ok 16 - len 6, expected len 6
+    ok 17 - mp_sizeof_datetime(6) == encoded length 6
+    ok 18 - mp_decode_datetime() return code
+    ok 19 - mp_sizeof_uuid() == decoded length
+    ok 20 - datetime_compare(&date, &ret)
+    ok 21 - len 3, expected len 3
+    ok 22 - mp_sizeof_datetime(3) == encoded length 3
+    ok 23 - mp_decode_datetime() return code
+    ok 24 - mp_sizeof_uuid() == decoded length
+    ok 25 - datetime_compare(&date, &ret)
+    ok 26 - len 6, expected len 6
+    ok 27 - mp_sizeof_datetime(6) == encoded length 6
+    ok 28 - mp_decode_datetime() return code
+    ok 29 - mp_sizeof_uuid() == decoded length
+    ok 30 - datetime_compare(&date, &ret)
+    ok 31 - len 6, expected len 6
+    ok 32 - mp_sizeof_datetime(6) == encoded length 6
+    ok 33 - mp_decode_datetime() return code
+    ok 34 - mp_sizeof_uuid() == decoded length
+    ok 35 - datetime_compare(&date, &ret)
+    ok 36 - len 6, expected len 6
+    ok 37 - mp_sizeof_datetime(6) == encoded length 6
+    ok 38 - mp_decode_datetime() return code
+    ok 39 - mp_sizeof_uuid() == decoded length
+    ok 40 - datetime_compare(&date, &ret)
+    ok 41 - len 6, expected len 6
+    ok 42 - mp_sizeof_datetime(6) == encoded length 6
+    ok 43 - mp_decode_datetime() return code
+    ok 44 - mp_sizeof_uuid() == decoded length
+    ok 45 - datetime_compare(&date, &ret)
+    ok 46 - len 9, expected len 9
+    ok 47 - mp_sizeof_datetime(9) == encoded length 9
+    ok 48 - mp_decode_datetime() return code
+    ok 49 - mp_sizeof_uuid() == decoded length
+    ok 50 - datetime_compare(&date, &ret)
+    ok 51 - len 9, expected len 9
+    ok 52 - mp_sizeof_datetime(9) == encoded length 9
+    ok 53 - mp_decode_datetime() return code
+    ok 54 - mp_sizeof_uuid() == decoded length
+    ok 55 - datetime_compare(&date, &ret)
+    ok 56 - len 9, expected len 9
+    ok 57 - mp_sizeof_datetime(9) == encoded length 9
+    ok 58 - mp_decode_datetime() return code
+    ok 59 - mp_sizeof_uuid() == decoded length
+    ok 60 - datetime_compare(&date, &ret)
+    ok 61 - len 8, expected len 8
+    ok 62 - mp_sizeof_datetime(8) == encoded length 8
+    ok 63 - mp_decode_datetime() return code
+    ok 64 - mp_sizeof_uuid() == decoded length
+    ok 65 - datetime_compare(&date, &ret)
+    ok 66 - len 8, expected len 8
+    ok 67 - mp_sizeof_datetime(8) == encoded length 8
+    ok 68 - mp_decode_datetime() return code
+    ok 69 - mp_sizeof_uuid() == decoded length
+    ok 70 - datetime_compare(&date, &ret)
+    ok 71 - len 12, expected len 12
+    ok 72 - mp_sizeof_datetime(12) == encoded length 12
+    ok 73 - mp_decode_datetime() return code
+    ok 74 - mp_sizeof_uuid() == decoded length
+    ok 75 - datetime_compare(&date, &ret)
+    ok 76 - len 17, expected len 17
+    ok 77 - mp_sizeof_datetime(17) == encoded length 17
+    ok 78 - mp_decode_datetime() return code
+    ok 79 - mp_sizeof_uuid() == decoded length
+    ok 80 - datetime_compare(&date, &ret)
+    ok 81 - len 18, expected len 18
+    ok 82 - mp_sizeof_datetime(18) == encoded length 18
+    ok 83 - mp_decode_datetime() return code
+    ok 84 - mp_sizeof_uuid() == decoded length
+    ok 85 - datetime_compare(&date, &ret)
+ok 3 - subtests
+    1..5
+	*** mp_print_test ***
+    ok 1 - correct mp_snprint size 17 with empty buffer
+    ok 2 - correct mp_snprint size 17
+    ok 3 - correct mp_snprint result
+    ok 4 - correct mp_fprint size 17
+    ok 5 - correct mp_fprint result 17
+	*** mp_print_test: done ***
+ok 4 - subtests
diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c
index 7a326075a..4d1b28ca1 100644
--- a/third_party/lua-cjson/lua_cjson.c
+++ b/third_party/lua-cjson/lua_cjson.c
@@ -52,6 +52,7 @@
 #include "mp_extension_types.h" /* MP_DECIMAL, MP_UUID */
 #include "tt_static.h"
 #include "uuid/tt_uuid.h" /* tt_uuid_to_string(), UUID_STR_LEN */
+#include "core/datetime.h"
 #include "cord_buf.h"
 
 typedef enum {
@@ -426,6 +427,13 @@ static void json_append_data(lua_State *l, struct luaL_serializer *cfg,
         case MP_UUID:
             return json_append_string(cfg, json, tt_uuid_str(field.uuidval),
                                       UUID_STR_LEN);
+
+        case MP_DATETIME:
+        {
+            char buf[DT_TO_STRING_BUFSIZE];
+            size_t sz = datetime_to_string(field.dateval, buf, sizeof(buf));
+            return json_append_string(cfg, json, buf, sz);
+        }
         default:
             assert(false);
         }
diff --git a/third_party/lua-yaml/lyaml.cc b/third_party/lua-yaml/lyaml.cc
index 2b67dcc6a..33c7cdfa1 100644
--- a/third_party/lua-yaml/lyaml.cc
+++ b/third_party/lua-yaml/lyaml.cc
@@ -617,7 +617,7 @@ static int dump_node(struct lua_yaml_dumper *dumper)
    yaml_event_t ev;
    yaml_scalar_style_t style = YAML_PLAIN_SCALAR_STYLE;
    int is_binary = 0;
-   char buf[FPCONV_G_FMT_BUFSIZE];
+   char buf[MAX(FPCONV_G_FMT_BUFSIZE, DT_TO_STRING_BUFSIZE)];
    struct luaL_field field;
    bool unused;
    (void) unused;
@@ -707,6 +707,10 @@ static int dump_node(struct lua_yaml_dumper *dumper)
          str = tt_uuid_str(field.uuidval);
          len = UUID_STR_LEN;
          break;
+      case MP_DATETIME:
+         len = datetime_to_string(field.dateval, buf, sizeof(buf));
+         str = buf;
+         break;
       default:
          assert(0); /* checked by luaL_checkfield() */
       }
-- 
2.29.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices
  2021-08-19  2:56 [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation Timur Safin via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime Timur Safin via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 2/5] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
@ 2021-08-19  2:56 ` Timur Safin via Tarantool-patches
  2021-08-19 10:16   ` Serge Petrenko via Tarantool-patches
  2021-08-19 11:18   ` UNera via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser Timur Safin via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 5/5] datetime: changelog for datetime module Timur Safin via Tarantool-patches
  4 siblings, 2 replies; 20+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-19  2:56 UTC (permalink / raw)
  To: vdavydov, sergepetrenko, tarantool-patches; +Cc: v.shpilevoy

* storage hints implemented for datetime_t values;
* proper comparison for indices of datetime type.

Part of #5941
Part of #5946

@TarantoolBot document

Title: Storage support for datetime values

It's now possible to store datetime values in spaces and create
indexed datetime fields.

Please refer to https://github.com/tarantool/tarantool/discussions/6244
for more detailed description of a storage schema.
---
 src/box/field_def.c           | 35 +++++++++-------
 src/box/field_def.h           |  1 +
 src/box/tuple_compare.cc      | 77 +++++++++++++++++++++++++++++++++++
 src/lib/core/datetime.c       |  9 ++++
 src/lib/core/datetime.h       | 11 +++++
 test/engine/datetime.result   | 77 +++++++++++++++++++++++++++++++++++
 test/engine/datetime.test.lua | 35 ++++++++++++++++
 test/unit/datetime.c          |  3 +-
 8 files changed, 232 insertions(+), 16 deletions(-)
 create mode 100644 test/engine/datetime.result
 create mode 100644 test/engine/datetime.test.lua

diff --git a/src/box/field_def.c b/src/box/field_def.c
index 51acb8025..cd5db942e 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -70,6 +70,7 @@ const uint32_t field_mp_type[] = {
 		(1U << MP_BIN) | (1U << MP_BOOL),
 	/* [FIELD_TYPE_DECIMAL]  =  */ 0, /* only MP_DECIMAL is supported */
 	/* [FIELD_TYPE_UUID]     =  */ 0, /* only MP_UUID is supported */
+	/* [FIELD_TYPE_DATETIME] =  */ 0, /* only MP_DATETIME is supported */
 	/* [FIELD_TYPE_ARRAY]    =  */ 1U << MP_ARRAY,
 	/* [FIELD_TYPE_MAP]      =  */ (1U << MP_MAP),
 };
@@ -83,9 +84,11 @@ const uint32_t field_ext_type[] = {
 	/* [FIELD_TYPE_INTEGER]   = */ 0,
 	/* [FIELD_TYPE_BOOLEAN]   = */ 0,
 	/* [FIELD_TYPE_VARBINARY] = */ 0,
-	/* [FIELD_TYPE_SCALAR]    = */ (1U << MP_DECIMAL) | (1U << MP_UUID),
+	/* [FIELD_TYPE_SCALAR]    = */ (1U << MP_DECIMAL) | (1U << MP_UUID) |
+		(1U << MP_DATETIME),
 	/* [FIELD_TYPE_DECIMAL]   = */ 1U << MP_DECIMAL,
 	/* [FIELD_TYPE_UUID]      = */ 1U << MP_UUID,
+	/* [FIELD_TYPE_DATETIME]  = */ 1U << MP_DATETIME,
 	/* [FIELD_TYPE_ARRAY]     = */ 0,
 	/* [FIELD_TYPE_MAP]       = */ 0,
 };
@@ -102,6 +105,7 @@ const char *field_type_strs[] = {
 	/* [FIELD_TYPE_SCALAR]   = */ "scalar",
 	/* [FIELD_TYPE_DECIMAL]  = */ "decimal",
 	/* [FIELD_TYPE_UUID]     = */ "uuid",
+	/* [FIELD_TYPE_DATETIME] = */ "datetime",
 	/* [FIELD_TYPE_ARRAY]    = */ "array",
 	/* [FIELD_TYPE_MAP]      = */ "map",
 };
@@ -128,20 +132,21 @@ field_type_by_name_wrapper(const char *str, uint32_t len)
  * values can be stored in the j type.
  */
 static const bool field_type_compatibility[] = {
-	   /*   ANY   UNSIGNED  STRING   NUMBER  DOUBLE  INTEGER  BOOLEAN VARBINARY SCALAR  DECIMAL   UUID    ARRAY    MAP  */
-/*   ANY    */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   false,
-/* UNSIGNED */ true,   true,    false,   true,    false,   true,    false,   false,  true,   false,  false,   false,   false,
-/*  STRING  */ true,   false,   true,    false,   false,   false,   false,   false,  true,   false,  false,   false,   false,
-/*  NUMBER  */ true,   false,   false,   true,    false,   false,   false,   false,  true,   false,  false,   false,   false,
-/*  DOUBLE  */ true,   false,   false,   true,    true,    false,   false,   false,  true,   false,  false,   false,   false,
-/*  INTEGER */ true,   false,   false,   true,    false,   true,    false,   false,  true,   false,  false,   false,   false,
-/*  BOOLEAN */ true,   false,   false,   false,   false,   false,   true,    false,  true,   false,  false,   false,   false,
-/* VARBINARY*/ true,   false,   false,   false,   false,   false,   false,   true,   true,   false,  false,   false,   false,
-/*  SCALAR  */ true,   false,   false,   false,   false,   false,   false,   false,  true,   false,  false,   false,   false,
-/*  DECIMAL */ true,   false,   false,   true,    false,   false,   false,   false,  true,   true,   false,   false,   false,
-/*   UUID   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  true,    false,   false,
-/*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   true,    false,
-/*    MAP   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,   true,
+	   /*   ANY   UNSIGNED  STRING   NUMBER  DOUBLE  INTEGER  BOOLEAN VARBINARY SCALAR  DECIMAL   UUID   DATETIME   ARRAY    MAP   */
+/*   ANY    */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,    false,   false,
+/* UNSIGNED */ true,   true,    false,   true,    false,   true,    false,   false,  true,   false,  false,   false,    false,   false,
+/*  STRING  */ true,   false,   true,    false,   false,   false,   false,   false,  true,   false,  false,   false,    false,   false,
+/*  NUMBER  */ true,   false,   false,   true,    false,   false,   false,   false,  true,   false,  false,   false,    false,   false,
+/*  DOUBLE  */ true,   false,   false,   true,    true,    false,   false,   false,  true,   false,  false,   false,    false,   false,
+/*  INTEGER */ true,   false,   false,   true,    false,   true,    false,   false,  true,   false,  false,   false,    false,   false,
+/*  BOOLEAN */ true,   false,   false,   false,   false,   false,   true,    false,  true,   false,  false,   false,    false,   false,
+/* VARBINARY*/ true,   false,   false,   false,   false,   false,   false,   true,   true,   false,  false,   false,    false,   false,
+/*  SCALAR  */ true,   false,   false,   false,   false,   false,   false,   false,  true,   false,  false,   false,    false,   false,
+/*  DECIMAL */ true,   false,   false,   true,    false,   false,   false,   false,  true,   true,   false,   false,    false,   false,
+/*   UUID   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  true,    false,    false,   false,
+/* DATETIME */ true,   false,   false,   false,   false,   false,   false,   false,  true,   false,  false,   true,     false,   false,
+/*   ARRAY  */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,    true,    false,
+/*    MAP   */ true,   false,   false,   false,   false,   false,   false,   false,  false,  false,  false,   false,    false,   true,
 };
 
 bool
diff --git a/src/box/field_def.h b/src/box/field_def.h
index c5cfe5e86..619ea19fe 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -61,6 +61,7 @@ enum field_type {
 	FIELD_TYPE_SCALAR,
 	FIELD_TYPE_DECIMAL,
 	FIELD_TYPE_UUID,
+	FIELD_TYPE_DATETIME,
 	FIELD_TYPE_ARRAY,
 	FIELD_TYPE_MAP,
 	field_type_MAX
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index 43cd29ce9..2478498ba 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -36,6 +36,7 @@
 #include "lib/core/decimal.h"
 #include "lib/core/mp_decimal.h"
 #include "uuid/mp_uuid.h"
+#include "lib/core/mp_datetime.h"
 #include "lib/core/mp_extension_types.h"
 
 /* {{{ tuple_compare */
@@ -76,6 +77,7 @@ enum mp_class {
 	MP_CLASS_STR,
 	MP_CLASS_BIN,
 	MP_CLASS_UUID,
+	MP_CLASS_DATETIME,
 	MP_CLASS_ARRAY,
 	MP_CLASS_MAP,
 	mp_class_max,
@@ -99,6 +101,8 @@ static enum mp_class mp_ext_classes[] = {
 	/* .MP_UNKNOWN_EXTENSION = */ mp_class_max, /* unsupported */
 	/* .MP_DECIMAL		 = */ MP_CLASS_NUMBER,
 	/* .MP_UUID		 = */ MP_CLASS_UUID,
+	/* .MP_ERROR		 = */ mp_class_max,
+	/* .MP_DATETIME		 = */ MP_CLASS_DATETIME,
 };
 
 static enum mp_class
@@ -390,6 +394,19 @@ mp_compare_uuid(const char *field_a, const char *field_b)
 	return memcmp(field_a + 2, field_b + 2, UUID_PACKED_LEN);
 }
 
+static int
+mp_compare_datetime(const char *lhs, const char *rhs)
+{
+	struct datetime lhs_dt, rhs_dt;
+	struct datetime *ret;
+	ret = mp_decode_datetime(&lhs, &lhs_dt);
+	assert(ret != NULL);
+	ret = mp_decode_datetime(&rhs, &rhs_dt);
+	assert(ret != NULL);
+	(void)ret;
+	return datetime_compare(&lhs_dt, &rhs_dt);
+}
+
 typedef int (*mp_compare_f)(const char *, const char *);
 static mp_compare_f mp_class_comparators[] = {
 	/* .MP_CLASS_NIL    = */ NULL,
@@ -398,6 +415,7 @@ static mp_compare_f mp_class_comparators[] = {
 	/* .MP_CLASS_STR    = */ mp_compare_str,
 	/* .MP_CLASS_BIN    = */ mp_compare_bin,
 	/* .MP_CLASS_UUID   = */ mp_compare_uuid,
+	/* .MP_CLASS_DATETIME=*/ mp_compare_datetime,
 	/* .MP_CLASS_ARRAY  = */ NULL,
 	/* .MP_CLASS_MAP    = */ NULL,
 };
@@ -478,6 +496,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
 		return mp_compare_decimal(field_a, field_b);
 	case FIELD_TYPE_UUID:
 		return mp_compare_uuid(field_a, field_b);
+	case FIELD_TYPE_DATETIME:
+		return mp_compare_datetime(field_a, field_b);
 	default:
 		unreachable();
 		return 0;
@@ -518,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
 						   field_b, b_type);
 	case FIELD_TYPE_UUID:
 		return mp_compare_uuid(field_a, field_b);
+	case FIELD_TYPE_DATETIME:
+		return mp_compare_datetime(field_a, field_b);
 	default:
 		unreachable();
 		return 0;
@@ -1518,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint,
 #define HINT_VALUE_DOUBLE_MAX	(exp2(HINT_VALUE_BITS - 1) - 1)
 #define HINT_VALUE_DOUBLE_MIN	(-exp2(HINT_VALUE_BITS - 1))
 
+/**
+ * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds
+ * into 60 bits of hint value. The idea is to represent wide enough
+ * years range, and leave the rest of bits occupied from nanoseconds part:
+ * - 36 bits is enough for time range of [208BC..4147]
+ * - for nanoseconds there is left 24 bits, which are MSB part of
+ *   32-bit value
+ */
+#define HINT_VALUE_SECS_BITS	36
+#define HINT_VALUE_NSEC_BITS	(HINT_VALUE_BITS - HINT_VALUE_SECS_BITS)
+#define HINT_VALUE_SECS_MAX	((1LL << (HINT_VALUE_SECS_BITS - 1)) - 1)
+#define HINT_VALUE_SECS_MIN	(-(1LL << (HINT_VALUE_SECS_BITS - 1)))
+#define HINT_VALUE_NSEC_SHIFT	(sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS)
+#define HINT_VALUE_NSEC_MAX	((1ULL << HINT_VALUE_NSEC_BITS) - 1)
+
 /*
  * HINT_CLASS_BITS should be big enough to store any mp_class value.
  * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE.
@@ -1610,6 +1647,25 @@ hint_uuid_raw(const char *data)
 	return hint_create(MP_CLASS_UUID, val);
 }
 
+static inline hint_t
+hint_datetime(struct datetime *date)
+{
+	/*
+	 * Use at most HINT_VALUE_SECS_BITS from datetime
+	 * seconds field as a hint value, and at MSB part
+	 * of HINT_VALUE_NSEC_BITS from nanoseconds.
+	 */
+	int64_t secs = date->secs;
+	int32_t nsec = date->nsec;
+	uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 :
+			secs - HINT_VALUE_SECS_MIN;
+	if (val >= HINT_VALUE_SECS_MAX)
+		val = HINT_VALUE_SECS_MAX;
+	val <<= HINT_VALUE_NSEC_BITS;
+	val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX;
+	return hint_create(MP_CLASS_DATETIME, val);
+}
+
 static inline uint64_t
 hint_str_raw(const char *s, uint32_t len)
 {
@@ -1741,6 +1797,17 @@ field_hint_uuid(const char *field)
 	return hint_uuid_raw(data);
 }
 
+static inline hint_t
+field_hint_datetime(const char *field)
+{
+	assert(mp_typeof(*field) == MP_EXT);
+	int8_t ext_type;
+	uint32_t len = mp_decode_extl(&field, &ext_type);
+	assert(ext_type == MP_DATETIME);
+	struct datetime date;
+	return hint_datetime(datetime_unpack(&field, len, &date));
+}
+
 static inline hint_t
 field_hint_string(const char *field, struct coll *coll)
 {
@@ -1792,6 +1859,11 @@ field_hint_scalar(const char *field, struct coll *coll)
 		}
 		case MP_UUID:
 			return hint_uuid_raw(field);
+		case MP_DATETIME:
+		{
+			struct datetime date;
+			return hint_datetime(datetime_unpack(&field, len, &date));
+		}
 		default:
 			unreachable();
 		}
@@ -1829,6 +1901,8 @@ field_hint(const char *field, struct coll *coll)
 		return field_hint_decimal(field);
 	case FIELD_TYPE_UUID:
 		return field_hint_uuid(field);
+	case FIELD_TYPE_DATETIME:
+		return field_hint_datetime(field);
 	default:
 		unreachable();
 	}
@@ -1943,6 +2017,9 @@ key_def_set_hint_func(struct key_def *def)
 	case FIELD_TYPE_UUID:
 		key_def_set_hint_func<FIELD_TYPE_UUID>(def);
 		break;
+	case FIELD_TYPE_DATETIME:
+		key_def_set_hint_func<FIELD_TYPE_DATETIME>(def);
+		break;
 	default:
 		/* Invalid key definition. */
 		def->key_hint = NULL;
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
index 7125090e6..ebc05e29c 100644
--- a/src/lib/core/datetime.c
+++ b/src/lib/core/datetime.c
@@ -123,3 +123,12 @@ datetime_to_string(const struct datetime *date, char *buf, int len)
 	return sz;
 }
 
+int
+datetime_compare(const struct datetime *lhs, const struct datetime *rhs)
+{
+	int result = COMPARE_RESULT(lhs->secs, rhs->secs);
+	if (result != 0)
+		return result;
+
+	return COMPARE_RESULT(lhs->nsec, rhs->nsec);
+}
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
index fb537e372..fc5717115 100644
--- a/src/lib/core/datetime.h
+++ b/src/lib/core/datetime.h
@@ -81,6 +81,17 @@ struct datetime_interval {
  */
 #define DT_TO_STRING_BUFSIZE   48
 
+/*
+ * Compare arguments of a datetime type
+ * @param lhs left datetime argument
+ * @param rhs right datetime argument
+ * @retval < 0 if lhs less than rhs
+ * @retval = 0 if lhs and rhs equal
+ * @retval > 0 if lhs greater than rhs
+ */
+int
+datetime_compare(const struct datetime *lhs, const struct datetime *rhs);
+
 /**
  * Convert datetime to string using default format
  * @param date source datetime value
diff --git a/test/engine/datetime.result b/test/engine/datetime.result
new file mode 100644
index 000000000..848a0aaec
--- /dev/null
+++ b/test/engine/datetime.result
@@ -0,0 +1,77 @@
+-- test-run result file version 2
+env = require('test_run')
+ | ---
+ | ...
+test_run = env.new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+
+date = require('datetime')
+ | ---
+ | ...
+
+_ = box.schema.space.create('T', {engine = engine})
+ | ---
+ | ...
+_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
+ | ---
+ | ...
+
+box.space.T:insert{date('1970-01-01')}\
+box.space.T:insert{date('1970-01-02')}\
+box.space.T:insert{date('1970-01-03')}\
+box.space.T:insert{date('2000-01-01')}
+ | ---
+ | ...
+
+o = box.space.T:select{}
+ | ---
+ | ...
+assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
+ | ---
+ | - true
+ | ...
+assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
+ | ---
+ | - true
+ | ...
+assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
+ | ---
+ | - true
+ | ...
+assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
+ | ---
+ | - true
+ | ...
+
+for i = 1,16 do\
+    box.space.T:insert{date.now()}\
+end
+ | ---
+ | ...
+
+a = box.space.T:select{}
+ | ---
+ | ...
+err = {}
+ | ---
+ | ...
+for i = 1, #a - 1 do\
+    if a[i][1] >= a[i+1][1] then\
+        table.insert(err, {a[i][1], a[i+1][1]})\
+        break\
+    end\
+end
+ | ---
+ | ...
+
+err
+ | ---
+ | - []
+ | ...
+box.space.T:drop()
+ | ---
+ | ...
diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua
new file mode 100644
index 000000000..3685e4d4b
--- /dev/null
+++ b/test/engine/datetime.test.lua
@@ -0,0 +1,35 @@
+env = require('test_run')
+test_run = env.new()
+engine = test_run:get_cfg('engine')
+
+date = require('datetime')
+
+_ = box.schema.space.create('T', {engine = engine})
+_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
+
+box.space.T:insert{date('1970-01-01')}\
+box.space.T:insert{date('1970-01-02')}\
+box.space.T:insert{date('1970-01-03')}\
+box.space.T:insert{date('2000-01-01')}
+
+o = box.space.T:select{}
+assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
+assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
+assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
+assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
+
+for i = 1,16 do\
+    box.space.T:insert{date.now()}\
+end
+
+a = box.space.T:select{}
+err = {}
+for i = 1, #a - 1 do\
+    if a[i][1] >= a[i+1][1] then\
+        table.insert(err, {a[i][1], a[i+1][1]})\
+        break\
+    end\
+end
+
+err
+box.space.T:drop()
diff --git a/test/unit/datetime.c b/test/unit/datetime.c
index c6ba444f5..61ebfe76b 100644
--- a/test/unit/datetime.c
+++ b/test/unit/datetime.c
@@ -281,7 +281,7 @@ mp_datetime_test()
 	};
 	size_t index;
 
-	plan(68);
+	plan(85);
 	for (index = 0; index < lengthof(tests); index++) {
 		struct datetime date = {
 			tests[index].secs,
@@ -303,6 +303,7 @@ mp_datetime_test()
 		struct datetime *rc = mp_decode_datetime(&data1, &ret);
 		is(rc, &ret, "mp_decode_datetime() return code");
 		is(data1, end, "mp_sizeof_uuid() == decoded length");
+		is(datetime_compare(&date, &ret), 0, "datetime_compare(&date, &ret)");
 	}
 	check_plan();
 }
-- 
2.29.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser
  2021-08-19  2:56 [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation Timur Safin via Tarantool-patches
                   ` (2 preceding siblings ...)
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
@ 2021-08-19  2:56 ` Timur Safin via Tarantool-patches
  2021-08-19 10:19   ` Serge Petrenko via Tarantool-patches
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 5/5] datetime: changelog for datetime module Timur Safin via Tarantool-patches
  4 siblings, 1 reply; 20+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-19  2:56 UTC (permalink / raw)
  To: vdavydov, sergepetrenko, tarantool-patches; +Cc: v.shpilevoy

It was told that if field `datetime.secs` would be `double` we should get
better performance in LuaJIT instead of `uint64_t` type, which is used at the
moment.

So we have created benchmark, which was comparing implementations of functions
from `datetime.c` if we would use `double` or `int64_t` for `datetime.secs` field.

Despite expectations, based on prior experience with floaing-point on x86
processors, comparison shows that `double` provides similar or
sometimes better timings. And picture stays consistent be it SSE2, AVX1 or
AVX2 code.

Part of #5941
---
 perf/CMakeLists.txt      |   3 +
 perf/datetime-common.h   | 105 +++++++++++++++++++
 perf/datetime-compare.cc | 213 +++++++++++++++++++++++++++++++++++++++
 perf/datetime-parser.cc  | 105 +++++++++++++++++++
 4 files changed, 426 insertions(+)
 create mode 100644 perf/datetime-common.h
 create mode 100644 perf/datetime-compare.cc
 create mode 100644 perf/datetime-parser.cc

diff --git a/perf/CMakeLists.txt b/perf/CMakeLists.txt
index 3651de5b4..b5d7caf81 100644
--- a/perf/CMakeLists.txt
+++ b/perf/CMakeLists.txt
@@ -12,3 +12,6 @@ include_directories(${CMAKE_SOURCE_DIR}/third_party)
 
 add_executable(tuple.perftest tuple.cc)
 target_link_libraries(tuple.perftest core box tuple benchmark::benchmark)
+
+add_executable(datetime.perftest datetime-parser.cc datetime-compare.cc)
+target_link_libraries(datetime.perftest cdt core benchmark::benchmark)
diff --git a/perf/datetime-common.h b/perf/datetime-common.h
new file mode 100644
index 000000000..6fd4e1e3b
--- /dev/null
+++ b/perf/datetime-common.h
@@ -0,0 +1,105 @@
+#include <assert.h>
+#include <stdint.h>
+#include <string.h>
+#include <benchmark/benchmark.h>
+
+#include "dt.h"
+#include "datetime.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+static const char sample[] = "2012-12-24T15:30Z";
+
+#define S(s)               \
+	{ s, sizeof(s) - 1 }
+
+static struct
+{
+	const char *sz;
+	size_t len;
+} tests[] = {
+    S("2012-12-24 15:30Z"),
+    S("2012-12-24 15:30z"),
+    S("2012-12-24 15:30"),
+    S("2012-12-24 16:30+01:00"),
+    S("2012-12-24 16:30+0100"),
+    S("2012-12-24 16:30+01"),
+    S("2012-12-24 14:30-01:00"),
+    S("2012-12-24 14:30-0100"),
+    S("2012-12-24 14:30-01"),
+    S("2012-12-24 15:30:00Z"),
+    S("2012-12-24 15:30:00z"),
+    S("2012-12-24 15:30:00"),
+    S("2012-12-24 16:30:00+01:00"),
+    S("2012-12-24 16:30:00+0100"),
+    S("2012-12-24 14:30:00-01:00"),
+    S("2012-12-24 14:30:00-0100"),
+    S("2012-12-24 15:30:00.123456Z"),
+    S("2012-12-24 15:30:00.123456z"),
+    S("2012-12-24 15:30:00.123456"),
+    S("2012-12-24 16:30:00.123456+01:00"),
+    S("2012-12-24 16:30:00.123456+01"),
+    S("2012-12-24 14:30:00.123456-01:00"),
+    S("2012-12-24 14:30:00.123456-01"),
+    S("2012-12-24t15:30Z"),
+    S("2012-12-24t15:30z"),
+    S("2012-12-24t15:30"),
+    S("2012-12-24t16:30+01:00"),
+    S("2012-12-24t16:30+0100"),
+    S("2012-12-24t14:30-01:00"),
+    S("2012-12-24t14:30-0100"),
+    S("2012-12-24t15:30:00Z"),
+    S("2012-12-24t15:30:00z"),
+    S("2012-12-24t15:30:00"),
+    S("2012-12-24t16:30:00+01:00"),
+    S("2012-12-24t16:30:00+0100"),
+    S("2012-12-24t14:30:00-01:00"),
+    S("2012-12-24t14:30:00-0100"),
+    S("2012-12-24t15:30:00.123456Z"),
+    S("2012-12-24t15:30:00.123456z"),
+    S("2012-12-24t16:30:00.123456+01:00"),
+    S("2012-12-24t14:30:00.123456-01:00"),
+    S("2012-12-24 16:30 +01:00"),
+    S("2012-12-24 14:30 -01:00"),
+    S("2012-12-24 15:30 UTC"),
+    S("2012-12-24 16:30 UTC+1"),
+    S("2012-12-24 16:30 UTC+01"),
+    S("2012-12-24 16:30 UTC+0100"),
+    S("2012-12-24 16:30 UTC+01:00"),
+    S("2012-12-24 14:30 UTC-1"),
+    S("2012-12-24 14:30 UTC-01"),
+    S("2012-12-24 14:30 UTC-01:00"),
+    S("2012-12-24 14:30 UTC-0100"),
+    S("2012-12-24 15:30 GMT"),
+    S("2012-12-24 16:30 GMT+1"),
+    S("2012-12-24 16:30 GMT+01"),
+    S("2012-12-24 16:30 GMT+0100"),
+    S("2012-12-24 16:30 GMT+01:00"),
+    S("2012-12-24 14:30 GMT-1"),
+    S("2012-12-24 14:30 GMT-01"),
+    S("2012-12-24 14:30 GMT-01:00"),
+    S("2012-12-24 14:30 GMT-0100"),
+    S("2012-12-24 14:30 -01:00"),
+    S("2012-12-24 16:30:00 +01:00"),
+    S("2012-12-24 14:30:00 -01:00"),
+    S("2012-12-24 16:30:00.123456 +01:00"),
+    S("2012-12-24 14:30:00.123456 -01:00"),
+    S("2012-12-24 15:30:00.123456 -00:00"),
+    S("20121224T1630+01:00"),
+    S("2012-12-24T1630+01:00"),
+    S("20121224T16:30+01"),
+    S("20121224T16:30 +01"),
+};
+#undef S
+
+#define DIM(a) (sizeof(a) / sizeof(a[0]))
+
+int
+parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
+	       int32_t *op);
+
+#ifdef __cplusplus
+}
+#endif
diff --git a/perf/datetime-compare.cc b/perf/datetime-compare.cc
new file mode 100644
index 000000000..5096eb987
--- /dev/null
+++ b/perf/datetime-compare.cc
@@ -0,0 +1,213 @@
+#include "dt.h"
+#include <string.h>
+#include <assert.h>
+#include <limits.h>
+
+#include "datetime-common.h"
+
+template <typename T>
+struct datetime_bench
+{
+	T secs;
+	uint32_t nsec;
+	uint32_t offset;
+
+static struct datetime_bench date_array[];
+};
+template<typename T>
+struct datetime_bench<T> datetime_bench<T>::date_array[DIM(tests)];
+
+/// Parse 70 datetime literals of various lengths
+template <typename T>
+static void
+Assign70()
+{
+	size_t index;
+	int64_t secs_expected;
+	int nanosecs;
+	int ofs;
+	using dt_bench = datetime_bench<T>;
+
+	for (index = 0; index < DIM(tests); index++) {
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+					&secs, &nanosecs, &ofs);
+		assert(rc == 0);
+		dt_bench::date_array[index].secs = (T)secs;
+		dt_bench::date_array[index].nsec = nanosecs;
+		dt_bench::date_array[index].offset = ofs;
+	}
+}
+
+template <typename T>
+static void
+DateTime_Assign70(benchmark::State &state)
+{
+	for (auto _ : state)
+		Assign70<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_Assign70, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_Assign70, double);
+
+#define COMPARE_RESULT_BENCH(a, b) (a < b ? -1 : a > b)
+
+template <typename T>
+int datetime_compare(const struct datetime_bench<T> *lhs,
+		     const struct datetime_bench<T> *rhs)
+{
+	int result = COMPARE_RESULT_BENCH(lhs->secs, rhs->secs);
+	if (result != 0)
+		return result;
+
+	return COMPARE_RESULT_BENCH(lhs->nsec, rhs->nsec);
+}
+
+template <typename T>
+static void
+AssignCompare70()
+{
+	size_t index;
+	int nanosecs;
+	int ofs;
+	using dt_bench = datetime_bench<T>;
+
+	size_t arrays_sz = DIM(tests);
+	for (index = 0; index < arrays_sz; index++) {
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+					&secs, &nanosecs, &ofs);
+		assert(rc == 0);
+		dt_bench::date_array[index].secs = (T)secs;
+		dt_bench::date_array[index].nsec = nanosecs;
+		dt_bench::date_array[index].offset = ofs;
+	}
+
+	for (index = 0; index < (arrays_sz - 1); index++) {
+		volatile int rc = datetime_compare<T>(&dt_bench::date_array[index],
+					     &dt_bench::date_array[index + 1]);
+		assert(rc == 0 || rc == -1 || rc == 1);
+	}
+}
+
+template <typename T>
+static void
+DateTime_AssignCompare70(benchmark::State &state)
+{
+	for (auto _ : state)
+		AssignCompare70<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_AssignCompare70, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_AssignCompare70, double);
+
+template <typename T>
+static void
+Compare20()
+{
+	size_t index;
+	int nanosecs;
+	int ofs;
+	using dt_bench = datetime_bench<T>;
+
+	for (size_t i = 0; i < 10; i++) {
+		volatile int rc = datetime_compare<T>(&dt_bench::date_array[i],
+					     &dt_bench::date_array[32 + i]);
+		assert(rc == 0 || rc == -1 || rc == 1);
+	}
+}
+
+template <typename T>
+static void
+DateTime_Compare20(benchmark::State &state)
+{
+	for (auto _ : state)
+		Compare20<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_Compare20, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_Compare20, double);
+
+
+#define SECS_EPOCH_1970_OFFSET ((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
+
+template<typename T>
+int
+datetime_to_string(const struct datetime_bench<T> *date, char *buf, uint32_t len)
+{
+#define ADVANCE(sz)		\
+	if (buf != NULL) { 	\
+		buf += sz; 	\
+		len -= sz; 	\
+	}			\
+	ret += sz;
+
+	int offset = date->offset;
+	/* for negative offsets around Epoch date we could get
+	 * negative secs value, which should be attributed to
+	 * 1969-12-31, not 1970-01-01, thus we first shift
+	 * epoch to Rata Die then divide by seconds per day,
+	 * not in reverse
+	 */
+	int64_t secs = (int64_t)date->secs + offset * 60 + SECS_EPOCH_1970_OFFSET;
+	assert((secs / SECS_PER_DAY) <= INT_MAX);
+	dt_t dt = dt_from_rdn(secs / SECS_PER_DAY);
+
+	int year, month, day, sec, ns, sign;
+	dt_to_ymd(dt, &year, &month, &day);
+
+	int hour = (secs / 3600) % 24,
+	    minute = (secs / 60) % 60;
+	sec = secs % 60;
+	ns = date->nsec;
+
+	int ret = 0;
+	uint32_t sz = snprintf(buf, len, "%04d-%02d-%02dT%02d:%02d",
+			       year, month, day, hour, minute);
+	ADVANCE(sz);
+	if (sec || ns) {
+		sz = snprintf(buf, len, ":%02d", sec);
+		ADVANCE(sz);
+		if (ns) {
+			if ((ns % 1000000) == 0)
+				sz = snprintf(buf, len, ".%03d", ns / 1000000);
+			else if ((ns % 1000) == 0)
+				sz = snprintf(buf, len, ".%06d", ns / 1000);
+			else
+				sz = snprintf(buf, len, ".%09d", ns);
+			ADVANCE(sz);
+		}
+	}
+	if (offset == 0) {
+		sz = snprintf(buf, len, "Z");
+		ADVANCE(sz);
+	}
+	else {
+		if (offset < 0)
+			sign = '-', offset = -offset;
+		else
+			sign = '+';
+
+		sz = snprintf(buf, len, "%c%02d:%02d", sign, offset / 60, offset % 60);
+		ADVANCE(sz);
+	}
+	return ret;
+}
+#undef ADVANCE
+
+template <typename T>
+static void
+ToString1()
+{
+	char buf[48];
+	struct datetime_bench<T> dateval = datetime_bench<T>::date_array[13];
+
+	volatile auto len = datetime_to_string<T>(&dateval, buf, sizeof(buf));
+}
+
+template <typename T>
+static void
+DateTime_ToString1(benchmark::State &state)
+{
+	for (auto _ : state)
+		ToString1<T>();
+}
+BENCHMARK_TEMPLATE1(DateTime_ToString1, uint64_t);
+BENCHMARK_TEMPLATE1(DateTime_ToString1, double);
diff --git a/perf/datetime-parser.cc b/perf/datetime-parser.cc
new file mode 100644
index 000000000..61557fe8f
--- /dev/null
+++ b/perf/datetime-parser.cc
@@ -0,0 +1,105 @@
+#include "dt.h"
+#include <string.h>
+#include <assert.h>
+
+#include "datetime-common.h"
+
+/* p5-time-moment/src/moment_parse.c: parse_string_lenient() */
+int
+parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
+	       int32_t *op)
+{
+	size_t n;
+	dt_t dt;
+	char c;
+	int sod = 0, nanosecond = 0, offset = 0;
+
+	n = dt_parse_iso_date(str, len, &dt);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	c = str[n++];
+	if (!(c == 'T' || c == 't' || c == ' '))
+		return 1;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_time(str, len, &sod, &nanosecond);
+	if (!n)
+		return 1;
+	if (n == len)
+		goto exit;
+
+	if (str[n] == ' ')
+	n++;
+
+	str += n;
+	len -= n;
+
+	n = dt_parse_iso_zone_lenient(str, len, &offset);
+	if (!n || n != len)
+		return 1;
+
+exit:
+	*sp = ((int64_t)dt_rdn(dt) - 719163) * 86400 + sod - offset * 60;
+	*np = nanosecond;
+	*op = offset;
+
+	return 0;
+}
+
+/// Parse 70 datetime literals of various lengths
+static void
+ParseTimeStamps()
+{
+	size_t index;
+	int64_t secs_expected;
+	int nanosecs;
+	int ofs;
+	parse_datetime(sample, sizeof(sample) - 1, &secs_expected,
+		       &nanosecs, &ofs);
+
+	for (index = 0; index < DIM(tests); index++)
+	{
+		int64_t secs;
+		int rc = parse_datetime(tests[index].sz, tests[index].len,
+					&secs, &nanosecs, &ofs);
+		assert(rc == 0);
+		assert(secs == secs_expected);
+	}
+}
+
+static void
+CDT_Parse70(benchmark::State &state)
+{
+	for (auto _ : state)
+		ParseTimeStamps();
+}
+BENCHMARK(CDT_Parse70);
+
+/// Parse single datetime literal of longest length
+static void
+Parse1()
+{
+	const char civil_string[] = "2015-02-18T10:50:31.521345123+10:00";
+	int64_t secs;
+	int nanosecs;
+	int ofs;
+	int rc = parse_datetime(civil_string, sizeof(civil_string) - 1,
+				&secs, &nanosecs, &ofs);
+	assert(rc == 0);
+	assert(nanosecs == 521345123);
+}
+
+static void
+CDT_Parse1(benchmark::State &state)
+{
+	for (auto _ : state)
+		Parse1();
+}
+BENCHMARK(CDT_Parse1);
+
+BENCHMARK_MAIN();
-- 
2.29.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [Tarantool-patches] [PATCH v6 5/5] datetime: changelog for datetime module
  2021-08-19  2:56 [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation Timur Safin via Tarantool-patches
                   ` (3 preceding siblings ...)
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser Timur Safin via Tarantool-patches
@ 2021-08-19  2:56 ` Timur Safin via Tarantool-patches
  2021-08-19 10:20   ` Serge Petrenko via Tarantool-patches
  4 siblings, 1 reply; 20+ messages in thread
From: Timur Safin via Tarantool-patches @ 2021-08-19  2:56 UTC (permalink / raw)
  To: vdavydov, sergepetrenko, tarantool-patches; +Cc: v.shpilevoy

Introduced new date/time/interval types support to lua and storage engines.

Closes #5941
Closes #5946
---
 changelogs/unreleased/gh-5941-datetime-type-support.md | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 changelogs/unreleased/gh-5941-datetime-type-support.md

diff --git a/changelogs/unreleased/gh-5941-datetime-type-support.md b/changelogs/unreleased/gh-5941-datetime-type-support.md
new file mode 100644
index 000000000..fb1f23077
--- /dev/null
+++ b/changelogs/unreleased/gh-5941-datetime-type-support.md
@@ -0,0 +1,6 @@
+## feature/lua/datetime
+
+ * Introduce new builtin module `datetime.lua` for date, time, and interval
+   support;
+ * Support of a new datetime type in storage engines allows to store datetime
+   fields and build indices with them (gh-5941, gh-5946).
-- 
2.29.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime Timur Safin via Tarantool-patches
@ 2021-08-19  9:43   ` Serge Petrenko via Tarantool-patches
  2021-08-19  9:47     ` Safin Timur via Tarantool-patches
  2021-08-19 15:26   ` Vladimir Davydov via Tarantool-patches
  1 sibling, 1 reply; 20+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-19  9:43 UTC (permalink / raw)
  To: Timur Safin, vdavydov, tarantool-patches; +Cc: v.shpilevoy



19.08.2021 05:56, Timur Safin пишет:
...
> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> new file mode 100644
> index 000000000..71feefded
> --- /dev/null
> +++ b/src/lib/core/datetime.h
> @@ -0,0 +1,86 @@
> +#pragma once
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + */
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include "c-dt/dt.h"
> +
> +#if defined(__cplusplus)
> +extern "C"
> +{
> +#endif /* defined(__cplusplus) */
> +
> +/**
> + * We count dates since so called "Rata Die" date
> + * January 1, 0001, Monday (as Day 1).
> + * But datetime structure keeps seconds since
> + * Unix "Epoch" date:
> + * Unix, January 1, 1970, Thursday
> + *
> + * The difference between Epoch (1970-01-01)
> + * and Rata Die (0001-01-01) is 719163 days.
> + */
> +
> +#ifndef SECS_PER_DAY
> +#define SECS_PER_DAY          86400
> +#define DT_EPOCH_1970_OFFSET  719163
> +#endif
> +
> +#define SECS_EPOCH_1970_OFFSET 	\
> +	((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
> +/**
> + * datetime structure keeps number of seconds since
> + * Unix Epoch.
> + * Time is normalized by UTC, so time-zone offset
> + * is informative only.
> + */
> +struct datetime {
> +	/** Seconds since Epoch. */
> +	double secs;
> +	/** Nanoseconds, if any. */
> +	uint32_t nsec;
> +	/** Offset in minutes from UTC. */
> +	int32_t offset;
> +};
> +
> +/**
> + * Date/time interval structure
> + */
> +struct datetime_interval {
> +	/** Relative seconds delta. */
> +	double secs;
> +	/** Nanoseconds delta, if any. */
> +	uint32_t nsec;
> +};
> +

Hi! Thanks for working on this!

I don't see you use this struct anywhere in the patch.
AFAICS interval operations are implemented via
add_years(), add_months() and so on.

Let's drop the struct then. We may introduce it once
it becomes necessary again.

Let's also drop the CTID_INTERVAL definition in lua/utils.c

LGTM, once you remove the interval.

> +/**
> + * Convert datetime to string using default format
> + * @param date source datetime value
> + * @param buf output character buffer
> + * @param len size ofoutput buffer
> + */
> +int
> +datetime_to_string(const struct datetime *date, char *buf, int len);
>
...

-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime
  2021-08-19  9:43   ` Serge Petrenko via Tarantool-patches
@ 2021-08-19  9:47     ` Safin Timur via Tarantool-patches
  0 siblings, 0 replies; 20+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-19  9:47 UTC (permalink / raw)
  To: Serge Petrenko, vdavydov, tarantool-patches; +Cc: v.shpilevoy


On 19.08.2021 12:43, Serge Petrenko wrote:
> 
> 
> 19.08.2021 05:56, Timur Safin пишет:
> ...
>> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
>> new file mode 100644
>> index 000000000..71feefded
>> --- /dev/null
>> +++ b/src/lib/core/datetime.h
>> @@ -0,0 +1,86 @@
>> +#pragma once
>> +/*
>> + * SPDX-License-Identifier: BSD-2-Clause
>> + *
>> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
>> + */
>> +
>> +#include <stdint.h>
>> +#include <stdbool.h>
>> +#include "c-dt/dt.h"
>> +
>> +#if defined(__cplusplus)
>> +extern "C"
>> +{
>> +#endif /* defined(__cplusplus) */
>> +
>> +/**
>> + * We count dates since so called "Rata Die" date
>> + * January 1, 0001, Monday (as Day 1).
>> + * But datetime structure keeps seconds since
>> + * Unix "Epoch" date:
>> + * Unix, January 1, 1970, Thursday
>> + *
>> + * The difference between Epoch (1970-01-01)
>> + * and Rata Die (0001-01-01) is 719163 days.
>> + */
>> +
>> +#ifndef SECS_PER_DAY
>> +#define SECS_PER_DAY          86400
>> +#define DT_EPOCH_1970_OFFSET  719163
>> +#endif
>> +
>> +#define SECS_EPOCH_1970_OFFSET     \
>> +    ((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
>> +/**
>> + * datetime structure keeps number of seconds since
>> + * Unix Epoch.
>> + * Time is normalized by UTC, so time-zone offset
>> + * is informative only.
>> + */
>> +struct datetime {
>> +    /** Seconds since Epoch. */
>> +    double secs;
>> +    /** Nanoseconds, if any. */
>> +    uint32_t nsec;
>> +    /** Offset in minutes from UTC. */
>> +    int32_t offset;
>> +};
>> +
>> +/**
>> + * Date/time interval structure
>> + */
>> +struct datetime_interval {
>> +    /** Relative seconds delta. */
>> +    double secs;
>> +    /** Nanoseconds delta, if any. */
>> +    uint32_t nsec;
>> +};
>> +
> 
> Hi! Thanks for working on this!
> 
> I don't see you use this struct anywhere in the patch.
> AFAICS interval operations are implemented via
> add_years(), add_months() and so on.
> 
> Let's drop the struct then. We may introduce it once
> it becomes necessary again.
> 
> Let's also drop the CTID_INTERVAL definition in lua/utils.c
> 
> LGTM, once you remove the interval.

Thanks for the spot - forgot to drop the C part. Here is incremental part.
------------------------------------------------
diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
index fc5717115..ab391a4f0 100644
--- a/src/lib/core/datetime.h
+++ b/src/lib/core/datetime.h
@@ -66,16 +66,6 @@ struct datetime {
  	int32_t offset;
  };

-/**
- * Date/time interval structure
- */
-struct datetime_interval {
-	/** Relative seconds delta. */
-	double secs;
-	/** Nanoseconds delta, if any. */
-	uint32_t nsec;
-};
-
  /**
   * Required size of datetime_to_string string buffer
   */
diff --git a/src/lua/utils.c b/src/lua/utils.c
index 2c89326f3..83e482b3d 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -49,7 +49,6 @@ uint32_t CTID_CHAR_PTR;
  uint32_t CTID_CONST_CHAR_PTR;
  uint32_t CTID_UUID;
  uint32_t CTID_DATETIME = 0;
-uint32_t CTID_INTERVAL = 0;


  void *
@@ -743,14 +742,6 @@ tarantool_lua_utils_init(struct lua_State *L)
  	(void) rc;
  	CTID_DATETIME = luaL_ctypeid(L, "struct datetime");
  	assert(CTID_DATETIME != 0);
-	rc = luaL_cdef(L, "struct datetime_interval {"
-			  "double secs;"
-			  "int32_t nsec;"
-			  "};");
-	assert(rc == 0);
-	(void) rc;
-	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval");
-	assert(CTID_INTERVAL != 0);

  	lua_pushcfunction(L, luaT_newthread_wrapper);
  	luaT_newthread_ref = luaL_ref(L, LUA_REGISTRYINDEX);
------------------------------------------------
> 
>> +/**
>> + * Convert datetime to string using default format
>> + * @param date source datetime value
>> + * @param buf output character buffer
>> + * @param len size ofoutput buffer
>> + */
>> +int
>> +datetime_to_string(const struct datetime *date, char *buf, int len);
>>
> ...
> 

Thanks,
Timur

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 2/5] box, datetime: messagepack support for datetime
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 2/5] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
@ 2021-08-19  9:58   ` Serge Petrenko via Tarantool-patches
  0 siblings, 0 replies; 20+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-19  9:58 UTC (permalink / raw)
  To: Timur Safin, vdavydov, tarantool-patches; +Cc: v.shpilevoy



19.08.2021 05:56, Timur Safin пишет:
> Serialize `struct datetime` as newly introduced MP_EXT type.
> It saves 1 required integer field and upto 2 optional
> unsigned fields in very compact fashion.
> - secs is required field;
> - but nsec, offset are both optional;
>
> * json, yaml serialization formats, lua output mode
>    supported;
> * exported symbols for datetime messagepack size calculations
>    so they are available for usage on Lua side.
>
> * As a bonus we introduce core/mp_utils.h with set of helpers
>    which simplify working with mp_sizeof*/mp_encode_*/mp_decode_*
>    functions regardless of a signedness of an integer data we
>    deal with.
>
> Part of #5941
> Part of #5946

Thanks! LGTM.


-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
@ 2021-08-19 10:16   ` Serge Petrenko via Tarantool-patches
  2021-08-19 11:18   ` UNera via Tarantool-patches
  1 sibling, 0 replies; 20+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-19 10:16 UTC (permalink / raw)
  To: Timur Safin, vdavydov, tarantool-patches; +Cc: v.shpilevoy



19.08.2021 05:56, Timur Safin пишет:
> * storage hints implemented for datetime_t values;
> * proper comparison for indices of datetime type.
>
> Part of #5941
> Part of #5946
>
> @TarantoolBot document
>
> Title: Storage support for datetime values
>
> It's now possible to store datetime values in spaces and create
> indexed datetime fields.
>
> Please refer to https://github.com/tarantool/tarantool/discussions/6244
> for more detailed description of a storage schema.
> ---

Thanks for the patch! LGTM.


-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser Timur Safin via Tarantool-patches
@ 2021-08-19 10:19   ` Serge Petrenko via Tarantool-patches
  2021-08-19 10:29     ` Safin Timur via Tarantool-patches
  0 siblings, 1 reply; 20+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-19 10:19 UTC (permalink / raw)
  To: Timur Safin, vdavydov, tarantool-patches; +Cc: v.shpilevoy



19.08.2021 05:56, Timur Safin пишет:
> It was told that if field `datetime.secs` would be `double` we should get
> better performance in LuaJIT instead of `uint64_t` type, which is used at the
> moment.
>
> So we have created benchmark, which was comparing implementations of functions
> from `datetime.c` if we would use `double` or `int64_t` for `datetime.secs` field.
>
> Despite expectations, based on prior experience with floaing-point on x86
> processors, comparison shows that `double` provides similar or
> sometimes better timings. And picture stays consistent be it SSE2, AVX1 or
> AVX2 code.
>
> Part of #5941
> ---

I agree with Vladimir here.
Looks like this perf test doesn't belong to Tarantool repository.
Would you mind dropping it?

>   perf/CMakeLists.txt      |   3 +
>   perf/datetime-common.h   | 105 +++++++++++++++++++
>   perf/datetime-compare.cc | 213 +++++++++++++++++++++++++++++++++++++++
>   perf/datetime-parser.cc  | 105 +++++++++++++++++++
>   4 files changed, 426 insertions(+)
>   create mode 100644 perf/datetime-common.h
>   create mode 100644 perf/datetime-compare.cc
>   create mode 100644 perf/datetime-parser.cc
>
> diff --git a/perf/CMakeLists.txt b/perf/CMakeLists.txt
> index 3651de5b4..b5d7caf81 100644
> --- a/perf/CMakeLists.txt
> +++ b/perf/CMakeLists.txt
> @@ -12,3 +12,6 @@ include_directories(${CMAKE_SOURCE_DIR}/third_party)
>   
>   add_executable(tuple.perftest tuple.cc)
>   target_link_libraries(tuple.perftest core box tuple benchmark::benchmark)
> +
> +add_executable(datetime.perftest datetime-parser.cc datetime-compare.cc)
> +target_link_libraries(datetime.perftest cdt core benchmark::benchmark)
> diff --git a/perf/datetime-common.h b/perf/datetime-common.h
> new file mode 100644
> index 000000000..6fd4e1e3b
> --- /dev/null
> +++ b/perf/datetime-common.h
> @@ -0,0 +1,105 @@
> +#include <assert.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <benchmark/benchmark.h>
> +
> +#include "dt.h"
> +#include "datetime.h"
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +static const char sample[] = "2012-12-24T15:30Z";
> +
> +#define S(s)               \
> +	{ s, sizeof(s) - 1 }
> +
> +static struct
> +{
> +	const char *sz;
> +	size_t len;
> +} tests[] = {
> +    S("2012-12-24 15:30Z"),
> +    S("2012-12-24 15:30z"),
> +    S("2012-12-24 15:30"),
> +    S("2012-12-24 16:30+01:00"),
> +    S("2012-12-24 16:30+0100"),
> +    S("2012-12-24 16:30+01"),
> +    S("2012-12-24 14:30-01:00"),
> +    S("2012-12-24 14:30-0100"),
> +    S("2012-12-24 14:30-01"),
> +    S("2012-12-24 15:30:00Z"),
> +    S("2012-12-24 15:30:00z"),
> +    S("2012-12-24 15:30:00"),
> +    S("2012-12-24 16:30:00+01:00"),
> +    S("2012-12-24 16:30:00+0100"),
> +    S("2012-12-24 14:30:00-01:00"),
> +    S("2012-12-24 14:30:00-0100"),
> +    S("2012-12-24 15:30:00.123456Z"),
> +    S("2012-12-24 15:30:00.123456z"),
> +    S("2012-12-24 15:30:00.123456"),
> +    S("2012-12-24 16:30:00.123456+01:00"),
> +    S("2012-12-24 16:30:00.123456+01"),
> +    S("2012-12-24 14:30:00.123456-01:00"),
> +    S("2012-12-24 14:30:00.123456-01"),
> +    S("2012-12-24t15:30Z"),
> +    S("2012-12-24t15:30z"),
> +    S("2012-12-24t15:30"),
> +    S("2012-12-24t16:30+01:00"),
> +    S("2012-12-24t16:30+0100"),
> +    S("2012-12-24t14:30-01:00"),
> +    S("2012-12-24t14:30-0100"),
> +    S("2012-12-24t15:30:00Z"),
> +    S("2012-12-24t15:30:00z"),
> +    S("2012-12-24t15:30:00"),
> +    S("2012-12-24t16:30:00+01:00"),
> +    S("2012-12-24t16:30:00+0100"),
> +    S("2012-12-24t14:30:00-01:00"),
> +    S("2012-12-24t14:30:00-0100"),
> +    S("2012-12-24t15:30:00.123456Z"),
> +    S("2012-12-24t15:30:00.123456z"),
> +    S("2012-12-24t16:30:00.123456+01:00"),
> +    S("2012-12-24t14:30:00.123456-01:00"),
> +    S("2012-12-24 16:30 +01:00"),
> +    S("2012-12-24 14:30 -01:00"),
> +    S("2012-12-24 15:30 UTC"),
> +    S("2012-12-24 16:30 UTC+1"),
> +    S("2012-12-24 16:30 UTC+01"),
> +    S("2012-12-24 16:30 UTC+0100"),
> +    S("2012-12-24 16:30 UTC+01:00"),
> +    S("2012-12-24 14:30 UTC-1"),
> +    S("2012-12-24 14:30 UTC-01"),
> +    S("2012-12-24 14:30 UTC-01:00"),
> +    S("2012-12-24 14:30 UTC-0100"),
> +    S("2012-12-24 15:30 GMT"),
> +    S("2012-12-24 16:30 GMT+1"),
> +    S("2012-12-24 16:30 GMT+01"),
> +    S("2012-12-24 16:30 GMT+0100"),
> +    S("2012-12-24 16:30 GMT+01:00"),
> +    S("2012-12-24 14:30 GMT-1"),
> +    S("2012-12-24 14:30 GMT-01"),
> +    S("2012-12-24 14:30 GMT-01:00"),
> +    S("2012-12-24 14:30 GMT-0100"),
> +    S("2012-12-24 14:30 -01:00"),
> +    S("2012-12-24 16:30:00 +01:00"),
> +    S("2012-12-24 14:30:00 -01:00"),
> +    S("2012-12-24 16:30:00.123456 +01:00"),
> +    S("2012-12-24 14:30:00.123456 -01:00"),
> +    S("2012-12-24 15:30:00.123456 -00:00"),
> +    S("20121224T1630+01:00"),
> +    S("2012-12-24T1630+01:00"),
> +    S("20121224T16:30+01"),
> +    S("20121224T16:30 +01"),
> +};
> +#undef S
> +
> +#define DIM(a) (sizeof(a) / sizeof(a[0]))
> +
> +int
> +parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
> +	       int32_t *op);
> +
> +#ifdef __cplusplus
> +}
> +#endif
> diff --git a/perf/datetime-compare.cc b/perf/datetime-compare.cc
> new file mode 100644
> index 000000000..5096eb987
> --- /dev/null
> +++ b/perf/datetime-compare.cc
> @@ -0,0 +1,213 @@
> +#include "dt.h"
> +#include <string.h>
> +#include <assert.h>
> +#include <limits.h>
> +
> +#include "datetime-common.h"
> +
> +template <typename T>
> +struct datetime_bench
> +{
> +	T secs;
> +	uint32_t nsec;
> +	uint32_t offset;
> +
> +static struct datetime_bench date_array[];
> +};
> +template<typename T>
> +struct datetime_bench<T> datetime_bench<T>::date_array[DIM(tests)];
> +
> +/// Parse 70 datetime literals of various lengths
> +template <typename T>
> +static void
> +Assign70()
> +{
> +	size_t index;
> +	int64_t secs_expected;
> +	int nanosecs;
> +	int ofs;
> +	using dt_bench = datetime_bench<T>;
> +
> +	for (index = 0; index < DIM(tests); index++) {
> +		int64_t secs;
> +		int rc = parse_datetime(tests[index].sz, tests[index].len,
> +					&secs, &nanosecs, &ofs);
> +		assert(rc == 0);
> +		dt_bench::date_array[index].secs = (T)secs;
> +		dt_bench::date_array[index].nsec = nanosecs;
> +		dt_bench::date_array[index].offset = ofs;
> +	}
> +}
> +
> +template <typename T>
> +static void
> +DateTime_Assign70(benchmark::State &state)
> +{
> +	for (auto _ : state)
> +		Assign70<T>();
> +}
> +BENCHMARK_TEMPLATE1(DateTime_Assign70, uint64_t);
> +BENCHMARK_TEMPLATE1(DateTime_Assign70, double);
> +
> +#define COMPARE_RESULT_BENCH(a, b) (a < b ? -1 : a > b)
> +
> +template <typename T>
> +int datetime_compare(const struct datetime_bench<T> *lhs,
> +		     const struct datetime_bench<T> *rhs)
> +{
> +	int result = COMPARE_RESULT_BENCH(lhs->secs, rhs->secs);
> +	if (result != 0)
> +		return result;
> +
> +	return COMPARE_RESULT_BENCH(lhs->nsec, rhs->nsec);
> +}
> +
> +template <typename T>
> +static void
> +AssignCompare70()
> +{
> +	size_t index;
> +	int nanosecs;
> +	int ofs;
> +	using dt_bench = datetime_bench<T>;
> +
> +	size_t arrays_sz = DIM(tests);
> +	for (index = 0; index < arrays_sz; index++) {
> +		int64_t secs;
> +		int rc = parse_datetime(tests[index].sz, tests[index].len,
> +					&secs, &nanosecs, &ofs);
> +		assert(rc == 0);
> +		dt_bench::date_array[index].secs = (T)secs;
> +		dt_bench::date_array[index].nsec = nanosecs;
> +		dt_bench::date_array[index].offset = ofs;
> +	}
> +
> +	for (index = 0; index < (arrays_sz - 1); index++) {
> +		volatile int rc = datetime_compare<T>(&dt_bench::date_array[index],
> +					     &dt_bench::date_array[index + 1]);
> +		assert(rc == 0 || rc == -1 || rc == 1);
> +	}
> +}
> +
> +template <typename T>
> +static void
> +DateTime_AssignCompare70(benchmark::State &state)
> +{
> +	for (auto _ : state)
> +		AssignCompare70<T>();
> +}
> +BENCHMARK_TEMPLATE1(DateTime_AssignCompare70, uint64_t);
> +BENCHMARK_TEMPLATE1(DateTime_AssignCompare70, double);
> +
> +template <typename T>
> +static void
> +Compare20()
> +{
> +	size_t index;
> +	int nanosecs;
> +	int ofs;
> +	using dt_bench = datetime_bench<T>;
> +
> +	for (size_t i = 0; i < 10; i++) {
> +		volatile int rc = datetime_compare<T>(&dt_bench::date_array[i],
> +					     &dt_bench::date_array[32 + i]);
> +		assert(rc == 0 || rc == -1 || rc == 1);
> +	}
> +}
> +
> +template <typename T>
> +static void
> +DateTime_Compare20(benchmark::State &state)
> +{
> +	for (auto _ : state)
> +		Compare20<T>();
> +}
> +BENCHMARK_TEMPLATE1(DateTime_Compare20, uint64_t);
> +BENCHMARK_TEMPLATE1(DateTime_Compare20, double);
> +
> +
> +#define SECS_EPOCH_1970_OFFSET ((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
> +
> +template<typename T>
> +int
> +datetime_to_string(const struct datetime_bench<T> *date, char *buf, uint32_t len)
> +{
> +#define ADVANCE(sz)		\
> +	if (buf != NULL) { 	\
> +		buf += sz; 	\
> +		len -= sz; 	\
> +	}			\
> +	ret += sz;
> +
> +	int offset = date->offset;
> +	/* for negative offsets around Epoch date we could get
> +	 * negative secs value, which should be attributed to
> +	 * 1969-12-31, not 1970-01-01, thus we first shift
> +	 * epoch to Rata Die then divide by seconds per day,
> +	 * not in reverse
> +	 */
> +	int64_t secs = (int64_t)date->secs + offset * 60 + SECS_EPOCH_1970_OFFSET;
> +	assert((secs / SECS_PER_DAY) <= INT_MAX);
> +	dt_t dt = dt_from_rdn(secs / SECS_PER_DAY);
> +
> +	int year, month, day, sec, ns, sign;
> +	dt_to_ymd(dt, &year, &month, &day);
> +
> +	int hour = (secs / 3600) % 24,
> +	    minute = (secs / 60) % 60;
> +	sec = secs % 60;
> +	ns = date->nsec;
> +
> +	int ret = 0;
> +	uint32_t sz = snprintf(buf, len, "%04d-%02d-%02dT%02d:%02d",
> +			       year, month, day, hour, minute);
> +	ADVANCE(sz);
> +	if (sec || ns) {
> +		sz = snprintf(buf, len, ":%02d", sec);
> +		ADVANCE(sz);
> +		if (ns) {
> +			if ((ns % 1000000) == 0)
> +				sz = snprintf(buf, len, ".%03d", ns / 1000000);
> +			else if ((ns % 1000) == 0)
> +				sz = snprintf(buf, len, ".%06d", ns / 1000);
> +			else
> +				sz = snprintf(buf, len, ".%09d", ns);
> +			ADVANCE(sz);
> +		}
> +	}
> +	if (offset == 0) {
> +		sz = snprintf(buf, len, "Z");
> +		ADVANCE(sz);
> +	}
> +	else {
> +		if (offset < 0)
> +			sign = '-', offset = -offset;
> +		else
> +			sign = '+';
> +
> +		sz = snprintf(buf, len, "%c%02d:%02d", sign, offset / 60, offset % 60);
> +		ADVANCE(sz);
> +	}
> +	return ret;
> +}
> +#undef ADVANCE
> +
> +template <typename T>
> +static void
> +ToString1()
> +{
> +	char buf[48];
> +	struct datetime_bench<T> dateval = datetime_bench<T>::date_array[13];
> +
> +	volatile auto len = datetime_to_string<T>(&dateval, buf, sizeof(buf));
> +}
> +
> +template <typename T>
> +static void
> +DateTime_ToString1(benchmark::State &state)
> +{
> +	for (auto _ : state)
> +		ToString1<T>();
> +}
> +BENCHMARK_TEMPLATE1(DateTime_ToString1, uint64_t);
> +BENCHMARK_TEMPLATE1(DateTime_ToString1, double);
> diff --git a/perf/datetime-parser.cc b/perf/datetime-parser.cc
> new file mode 100644
> index 000000000..61557fe8f
> --- /dev/null
> +++ b/perf/datetime-parser.cc
> @@ -0,0 +1,105 @@
> +#include "dt.h"
> +#include <string.h>
> +#include <assert.h>
> +
> +#include "datetime-common.h"
> +
> +/* p5-time-moment/src/moment_parse.c: parse_string_lenient() */
> +int
> +parse_datetime(const char *str, size_t len, int64_t *sp, int32_t *np,
> +	       int32_t *op)
> +{
> +	size_t n;
> +	dt_t dt;
> +	char c;
> +	int sod = 0, nanosecond = 0, offset = 0;
> +
> +	n = dt_parse_iso_date(str, len, &dt);
> +	if (!n)
> +		return 1;
> +	if (n == len)
> +		goto exit;
> +
> +	c = str[n++];
> +	if (!(c == 'T' || c == 't' || c == ' '))
> +		return 1;
> +
> +	str += n;
> +	len -= n;
> +
> +	n = dt_parse_iso_time(str, len, &sod, &nanosecond);
> +	if (!n)
> +		return 1;
> +	if (n == len)
> +		goto exit;
> +
> +	if (str[n] == ' ')
> +	n++;
> +
> +	str += n;
> +	len -= n;
> +
> +	n = dt_parse_iso_zone_lenient(str, len, &offset);
> +	if (!n || n != len)
> +		return 1;
> +
> +exit:
> +	*sp = ((int64_t)dt_rdn(dt) - 719163) * 86400 + sod - offset * 60;
> +	*np = nanosecond;
> +	*op = offset;
> +
> +	return 0;
> +}
> +
> +/// Parse 70 datetime literals of various lengths
> +static void
> +ParseTimeStamps()
> +{
> +	size_t index;
> +	int64_t secs_expected;
> +	int nanosecs;
> +	int ofs;
> +	parse_datetime(sample, sizeof(sample) - 1, &secs_expected,
> +		       &nanosecs, &ofs);
> +
> +	for (index = 0; index < DIM(tests); index++)
> +	{
> +		int64_t secs;
> +		int rc = parse_datetime(tests[index].sz, tests[index].len,
> +					&secs, &nanosecs, &ofs);
> +		assert(rc == 0);
> +		assert(secs == secs_expected);
> +	}
> +}
> +
> +static void
> +CDT_Parse70(benchmark::State &state)
> +{
> +	for (auto _ : state)
> +		ParseTimeStamps();
> +}
> +BENCHMARK(CDT_Parse70);
> +
> +/// Parse single datetime literal of longest length
> +static void
> +Parse1()
> +{
> +	const char civil_string[] = "2015-02-18T10:50:31.521345123+10:00";
> +	int64_t secs;
> +	int nanosecs;
> +	int ofs;
> +	int rc = parse_datetime(civil_string, sizeof(civil_string) - 1,
> +				&secs, &nanosecs, &ofs);
> +	assert(rc == 0);
> +	assert(nanosecs == 521345123);
> +}
> +
> +static void
> +CDT_Parse1(benchmark::State &state)
> +{
> +	for (auto _ : state)
> +		Parse1();
> +}
> +BENCHMARK(CDT_Parse1);
> +
> +BENCHMARK_MAIN();

-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 5/5] datetime: changelog for datetime module
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 5/5] datetime: changelog for datetime module Timur Safin via Tarantool-patches
@ 2021-08-19 10:20   ` Serge Petrenko via Tarantool-patches
  0 siblings, 0 replies; 20+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-19 10:20 UTC (permalink / raw)
  To: Timur Safin, vdavydov, tarantool-patches; +Cc: v.shpilevoy



19.08.2021 05:56, Timur Safin пишет:
> Introduced new date/time/interval types support to lua and storage engines.
>
> Closes #5941
> Closes #5946
> ---
>   changelogs/unreleased/gh-5941-datetime-type-support.md | 6 ++++++
>   1 file changed, 6 insertions(+)
>   create mode 100644 changelogs/unreleased/gh-5941-datetime-type-support.md
>
> diff --git a/changelogs/unreleased/gh-5941-datetime-type-support.md b/changelogs/unreleased/gh-5941-datetime-type-support.md
> new file mode 100644
> index 000000000..fb1f23077
> --- /dev/null
> +++ b/changelogs/unreleased/gh-5941-datetime-type-support.md
> @@ -0,0 +1,6 @@
> +## feature/lua/datetime
> +
> + * Introduce new builtin module `datetime.lua` for date, time, and interval
> +   support;
> + * Support of a new datetime type in storage engines allows to store datetime
> +   fields and build indices with them (gh-5941, gh-5946).
LGTM.

-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser
  2021-08-19 10:19   ` Serge Petrenko via Tarantool-patches
@ 2021-08-19 10:29     ` Safin Timur via Tarantool-patches
  2021-08-19 11:11       ` Serge Petrenko via Tarantool-patches
  2021-08-19 15:58       ` Vladimir Davydov via Tarantool-patches
  0 siblings, 2 replies; 20+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-19 10:29 UTC (permalink / raw)
  To: Serge Petrenko, vdavydov, tarantool-patches; +Cc: v.shpilevoy

On 19.08.2021 13:19, Serge Petrenko wrote:
> 
> 
> 19.08.2021 05:56, Timur Safin пишет:
>> It was told that if field `datetime.secs` would be `double` we should get
>> better performance in LuaJIT instead of `uint64_t` type, which is used 
>> at the
>> moment.
>>
>> So we have created benchmark, which was comparing implementations of 
>> functions
>> from `datetime.c` if we would use `double` or `int64_t` for 
>> `datetime.secs` field.
>>
>> Despite expectations, based on prior experience with floaing-point on x86
>> processors, comparison shows that `double` provides similar or
>> sometimes better timings. And picture stays consistent be it SSE2, 
>> AVX1 or
>> AVX2 code.
>>
>> Part of #5941
>> ---
> 
> I agree with Vladimir here.
> Looks like this perf test doesn't belong to Tarantool repository.
> Would you mind dropping it?

Here is the case (we both aware of) I want to avoid here - today we do 
not have saved _that_ decimal perf test, basing on which we have 
preferred LuaC and dropped FFI implementation. We could not rerun it 
today, within a newer LuaJIT implementation, to verify that situation 
didn't change. This is similar case - we have made a decision basing on 
some evaluations using this code, in a future we may decide to further 
optimize data structure (like Vova suggested elsewhere to split int64 
into 2 fields), and it would be better if at that moment we would still 
have performance test around for adaptations and rerun.

Yes, it's another test of performance test we used to see in perf 
directory (hehe, there is only single test at the moment), kind of one 
time shot in a history, important for design decision, but from longer 
prospective I assume it should be still around.

Does my reasoning make some sense?

Thanks,
Timur

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser
  2021-08-19 10:29     ` Safin Timur via Tarantool-patches
@ 2021-08-19 11:11       ` Serge Petrenko via Tarantool-patches
  2021-08-19 15:58       ` Vladimir Davydov via Tarantool-patches
  1 sibling, 0 replies; 20+ messages in thread
From: Serge Petrenko via Tarantool-patches @ 2021-08-19 11:11 UTC (permalink / raw)
  To: Safin Timur, vdavydov, tarantool-patches; +Cc: v.shpilevoy



19.08.2021 13:29, Safin Timur пишет:
> On 19.08.2021 13:19, Serge Petrenko wrote:
>>
>>
>> 19.08.2021 05:56, Timur Safin пишет:
>>> It was told that if field `datetime.secs` would be `double` we 
>>> should get
>>> better performance in LuaJIT instead of `uint64_t` type, which is 
>>> used at the
>>> moment.
>>>
>>> So we have created benchmark, which was comparing implementations of 
>>> functions
>>> from `datetime.c` if we would use `double` or `int64_t` for 
>>> `datetime.secs` field.
>>>
>>> Despite expectations, based on prior experience with floaing-point 
>>> on x86
>>> processors, comparison shows that `double` provides similar or
>>> sometimes better timings. And picture stays consistent be it SSE2, 
>>> AVX1 or
>>> AVX2 code.
>>>
>>> Part of #5941
>>> ---
>>
>> I agree with Vladimir here.
>> Looks like this perf test doesn't belong to Tarantool repository.
>> Would you mind dropping it?
>
> Here is the case (we both aware of) I want to avoid here - today we do 
> not have saved _that_ decimal perf test, basing on which we have 
> preferred LuaC and dropped FFI implementation. We could not rerun it 
> today, within a newer LuaJIT implementation, to verify that situation 
> didn't change. This is similar case - we have made a decision basing 
> on some evaluations using this code, in a future we may decide to 
> further optimize data structure (like Vova suggested elsewhere to 
> split int64 into 2 fields), and it would be better if at that moment 
> we would still have performance test around for adaptations and rerun.
>
> Yes, it's another test of performance test we used to see in perf 
> directory (hehe, there is only single test at the moment), kind of one 
> time shot in a history, important for design decision, but from longer 
> prospective I assume it should be still around.
>
> Does my reasoning make some sense?

Yes, I see your point.

But you've saved the test in some github discussion, AFAIR. I feel like 
it's an ok place to save it.

If you really want to save the test in this repo, maybe introduce a new 
folder?
Since perf/ is really intended to store perf tests. Not the ones that 
have influenced this or that
decision, but the ones that should be run every now and then to find 
regressions.

>
> Thanks,
> Timur

-- 
Serge Petrenko


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches]  [PATCH v6 3/5] box, datetime: datetime comparison for indices
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
  2021-08-19 10:16   ` Serge Petrenko via Tarantool-patches
@ 2021-08-19 11:18   ` UNera via Tarantool-patches
  2021-08-19 11:53     ` Safin Timur via Tarantool-patches
  1 sibling, 1 reply; 20+ messages in thread
From: UNera via Tarantool-patches @ 2021-08-19 11:18 UTC (permalink / raw)
  To: Timur Safin; +Cc: tarantool-patches, v.shpilevoy

[-- Attachment #1: Type: text/plain, Size: 17791 bytes --]


Hi!
 
I looked trhough the patchset.
 
I also compiled the bracnch and tested by hand.
 
Example:
 
tarantool> datetime = require('datetime')
---
...
tarantool> datetime.new('2020-01-02 02:00')
---
- 1970-01-01T00:00Z
...
tarantool> datetime.new('2020-01-02 02:00+0300')
---
- 1970-01-01T00:00Z
...
tarantool> datetime.new('2020-01-02 02:00:11+0300')
---
- 1970-01-01T00:00Z
...
tarantool> datetime.new('2020-01-02T02:00:11+0300')
---
- 1970-01-01T00:00Z
...
 
 
The code looks like as broken.
 
Also my notes:
 
*  tostring has to return datetime string with seconds and timezone
*  also the other databases try to avoid to use «Z» symbol in datetime-format, so I thing we should use «+00:00» instead «Z»
 
Yesterday we approved the following API:
 
T:add{year=XXX, month=YYY, ...}
T:sub{year=XXX, month=YYY, ...}
T:set{year=XXX, …, tz=’+03:00’}
T:strftime(‘%F %D %z’)
 
I want to see a test set for the API.

Also we approved the following methods:
  T:sec() — 35
T:min() — 12
T:day() — 19
T:wday() — 5
T:yday() — 231
T:year() — 2021
T:month() — 8
T:hour() — 23
T:tz() — ‘+03:00’
 
It would be nice to see test set for each of the functions.
 
Also it would be nice to rename ambigouos items «secs» to «epoch» or «timestamp»
Datetime object must have methods to return integer and float timestamp:
 
T:epoch()
T:timestamp()
  
>Четверг, 19 августа 2021, 5:58 +03:00 от Timur Safin via Tarantool-patches <tarantool-patches@dev.tarantool.org>:
> 
>* storage hints implemented for datetime_t values;
>* proper comparison for indices of datetime type.
>
>Part of #5941
>Part of #5946
>
>@TarantoolBot document
>
>Title: Storage support for datetime values
>
>It's now possible to store datetime values in spaces and create
>indexed datetime fields.
>
>Please refer to  https://github.com/tarantool/tarantool/discussions/6244
>for more detailed description of a storage schema.
>---
> src/box/field_def.c | 35 +++++++++-------
> src/box/field_def.h | 1 +
> src/box/tuple_compare.cc | 77 +++++++++++++++++++++++++++++++++++
> src/lib/core/datetime.c | 9 ++++
> src/lib/core/datetime.h | 11 +++++
> test/engine/datetime.result | 77 +++++++++++++++++++++++++++++++++++
> test/engine/datetime.test.lua | 35 ++++++++++++++++
> test/unit/datetime.c | 3 +-
> 8 files changed, 232 insertions(+), 16 deletions(-)
> create mode 100644 test/engine/datetime.result
> create mode 100644 test/engine/datetime.test.lua
>
>diff --git a/src/box/field_def.c b/src/box/field_def.c
>index 51acb8025..cd5db942e 100644
>--- a/src/box/field_def.c
>+++ b/src/box/field_def.c
>@@ -70,6 +70,7 @@ const uint32_t field_mp_type[] = {
>  (1U << MP_BIN) | (1U << MP_BOOL),
>  /* [FIELD_TYPE_DECIMAL] = */ 0, /* only MP_DECIMAL is supported */
>  /* [FIELD_TYPE_UUID] = */ 0, /* only MP_UUID is supported */
>+ /* [FIELD_TYPE_DATETIME] = */ 0, /* only MP_DATETIME is supported */
>  /* [FIELD_TYPE_ARRAY] = */ 1U << MP_ARRAY,
>  /* [FIELD_TYPE_MAP] = */ (1U << MP_MAP),
> };
>@@ -83,9 +84,11 @@ const uint32_t field_ext_type[] = {
>  /* [FIELD_TYPE_INTEGER] = */ 0,
>  /* [FIELD_TYPE_BOOLEAN] = */ 0,
>  /* [FIELD_TYPE_VARBINARY] = */ 0,
>- /* [FIELD_TYPE_SCALAR] = */ (1U << MP_DECIMAL) | (1U << MP_UUID),
>+ /* [FIELD_TYPE_SCALAR] = */ (1U << MP_DECIMAL) | (1U << MP_UUID) |
>+ (1U << MP_DATETIME),
>  /* [FIELD_TYPE_DECIMAL] = */ 1U << MP_DECIMAL,
>  /* [FIELD_TYPE_UUID] = */ 1U << MP_UUID,
>+ /* [FIELD_TYPE_DATETIME] = */ 1U << MP_DATETIME,
>  /* [FIELD_TYPE_ARRAY] = */ 0,
>  /* [FIELD_TYPE_MAP] = */ 0,
> };
>@@ -102,6 +105,7 @@ const char *field_type_strs[] = {
>  /* [FIELD_TYPE_SCALAR] = */ "scalar",
>  /* [FIELD_TYPE_DECIMAL] = */ "decimal",
>  /* [FIELD_TYPE_UUID] = */ "uuid",
>+ /* [FIELD_TYPE_DATETIME] = */ "datetime",
>  /* [FIELD_TYPE_ARRAY] = */ "array",
>  /* [FIELD_TYPE_MAP] = */ "map",
> };
>@@ -128,20 +132,21 @@ field_type_by_name_wrapper(const char *str, uint32_t len)
>  * values can be stored in the j type.
>  */
> static const bool field_type_compatibility[] = {
>- /* ANY UNSIGNED STRING NUMBER DOUBLE INTEGER BOOLEAN VARBINARY SCALAR DECIMAL UUID ARRAY MAP */
>-/* ANY */ true, false, false, false, false, false, false, false, false, false, false, false, false,
>-/* UNSIGNED */ true, true, false, true, false, true, false, false, true, false, false, false, false,
>-/* STRING */ true, false, true, false, false, false, false, false, true, false, false, false, false,
>-/* NUMBER */ true, false, false, true, false, false, false, false, true, false, false, false, false,
>-/* DOUBLE */ true, false, false, true, true, false, false, false, true, false, false, false, false,
>-/* INTEGER */ true, false, false, true, false, true, false, false, true, false, false, false, false,
>-/* BOOLEAN */ true, false, false, false, false, false, true, false, true, false, false, false, false,
>-/* VARBINARY*/ true, false, false, false, false, false, false, true, true, false, false, false, false,
>-/* SCALAR */ true, false, false, false, false, false, false, false, true, false, false, false, false,
>-/* DECIMAL */ true, false, false, true, false, false, false, false, true, true, false, false, false,
>-/* UUID */ true, false, false, false, false, false, false, false, false, false, true, false, false,
>-/* ARRAY */ true, false, false, false, false, false, false, false, false, false, false, true, false,
>-/* MAP */ true, false, false, false, false, false, false, false, false, false, false, false, true,
>+ /* ANY UNSIGNED STRING NUMBER DOUBLE INTEGER BOOLEAN VARBINARY SCALAR DECIMAL UUID DATETIME ARRAY MAP */
>+/* ANY */ true, false, false, false, false, false, false, false, false, false, false, false, false, false,
>+/* UNSIGNED */ true, true, false, true, false, true, false, false, true, false, false, false, false, false,
>+/* STRING */ true, false, true, false, false, false, false, false, true, false, false, false, false, false,
>+/* NUMBER */ true, false, false, true, false, false, false, false, true, false, false, false, false, false,
>+/* DOUBLE */ true, false, false, true, true, false, false, false, true, false, false, false, false, false,
>+/* INTEGER */ true, false, false, true, false, true, false, false, true, false, false, false, false, false,
>+/* BOOLEAN */ true, false, false, false, false, false, true, false, true, false, false, false, false, false,
>+/* VARBINARY*/ true, false, false, false, false, false, false, true, true, false, false, false, false, false,
>+/* SCALAR */ true, false, false, false, false, false, false, false, true, false, false, false, false, false,
>+/* DECIMAL */ true, false, false, true, false, false, false, false, true, true, false, false, false, false,
>+/* UUID */ true, false, false, false, false, false, false, false, false, false, true, false, false, false,
>+/* DATETIME */ true, false, false, false, false, false, false, false, true, false, false, true, false, false,
>+/* ARRAY */ true, false, false, false, false, false, false, false, false, false, false, false, true, false,
>+/* MAP */ true, false, false, false, false, false, false, false, false, false, false, false, false, true,
> };
> 
> bool
>diff --git a/src/box/field_def.h b/src/box/field_def.h
>index c5cfe5e86..619ea19fe 100644
>--- a/src/box/field_def.h
>+++ b/src/box/field_def.h
>@@ -61,6 +61,7 @@ enum field_type {
>  FIELD_TYPE_SCALAR,
>  FIELD_TYPE_DECIMAL,
>  FIELD_TYPE_UUID,
>+ FIELD_TYPE_DATETIME,
>  FIELD_TYPE_ARRAY,
>  FIELD_TYPE_MAP,
>  field_type_MAX
>diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
>index 43cd29ce9..2478498ba 100644
>--- a/src/box/tuple_compare.cc
>+++ b/src/box/tuple_compare.cc
>@@ -36,6 +36,7 @@
> #include "lib/core/decimal.h"
> #include "lib/core/mp_decimal.h"
> #include "uuid/mp_uuid.h"
>+#include "lib/core/mp_datetime.h"
> #include "lib/core/mp_extension_types.h"
> 
> /* {{{ tuple_compare */
>@@ -76,6 +77,7 @@ enum mp_class {
>  MP_CLASS_STR,
>  MP_CLASS_BIN,
>  MP_CLASS_UUID,
>+ MP_CLASS_DATETIME,
>  MP_CLASS_ARRAY,
>  MP_CLASS_MAP,
>  mp_class_max,
>@@ -99,6 +101,8 @@ static enum mp_class mp_ext_classes[] = {
>  /* .MP_UNKNOWN_EXTENSION = */ mp_class_max, /* unsupported */
>  /* .MP_DECIMAL = */ MP_CLASS_NUMBER,
>  /* .MP_UUID = */ MP_CLASS_UUID,
>+ /* .MP_ERROR = */ mp_class_max,
>+ /* .MP_DATETIME = */ MP_CLASS_DATETIME,
> };
> 
> static enum mp_class
>@@ -390,6 +394,19 @@ mp_compare_uuid(const char *field_a, const char *field_b)
>  return memcmp(field_a + 2, field_b + 2, UUID_PACKED_LEN);
> }
> 
>+static int
>+mp_compare_datetime(const char *lhs, const char *rhs)
>+{
>+ struct datetime lhs_dt, rhs_dt;
>+ struct datetime *ret;
>+ ret = mp_decode_datetime(&lhs, &lhs_dt);
>+ assert(ret != NULL);
>+ ret = mp_decode_datetime(&rhs, &rhs_dt);
>+ assert(ret != NULL);
>+ (void)ret;
>+ return datetime_compare(&lhs_dt, &rhs_dt);
>+}
>+
> typedef int (*mp_compare_f)(const char *, const char *);
> static mp_compare_f mp_class_comparators[] = {
>  /* .MP_CLASS_NIL = */ NULL,
>@@ -398,6 +415,7 @@ static mp_compare_f mp_class_comparators[] = {
>  /* .MP_CLASS_STR = */ mp_compare_str,
>  /* .MP_CLASS_BIN = */ mp_compare_bin,
>  /* .MP_CLASS_UUID = */ mp_compare_uuid,
>+ /* .MP_CLASS_DATETIME=*/ mp_compare_datetime,
>  /* .MP_CLASS_ARRAY = */ NULL,
>  /* .MP_CLASS_MAP = */ NULL,
> };
>@@ -478,6 +496,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
>  return mp_compare_decimal(field_a, field_b);
>  case FIELD_TYPE_UUID:
>  return mp_compare_uuid(field_a, field_b);
>+ case FIELD_TYPE_DATETIME:
>+ return mp_compare_datetime(field_a, field_b);
>  default:
>  unreachable();
>  return 0;
>@@ -518,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
>  field_b, b_type);
>  case FIELD_TYPE_UUID:
>  return mp_compare_uuid(field_a, field_b);
>+ case FIELD_TYPE_DATETIME:
>+ return mp_compare_datetime(field_a, field_b);
>  default:
>  unreachable();
>  return 0;
>@@ -1518,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint,
> #define HINT_VALUE_DOUBLE_MAX (exp2(HINT_VALUE_BITS - 1) - 1)
> #define HINT_VALUE_DOUBLE_MIN (-exp2(HINT_VALUE_BITS - 1))
> 
>+/**
>+ * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds
>+ * into 60 bits of hint value. The idea is to represent wide enough
>+ * years range, and leave the rest of bits occupied from nanoseconds part:
>+ * - 36 bits is enough for time range of [208BC..4147]
>+ * - for nanoseconds there is left 24 bits, which are MSB part of
>+ * 32-bit value
>+ */
>+#define HINT_VALUE_SECS_BITS 36
>+#define HINT_VALUE_NSEC_BITS (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS)
>+#define HINT_VALUE_SECS_MAX ((1LL << (HINT_VALUE_SECS_BITS - 1)) - 1)
>+#define HINT_VALUE_SECS_MIN (-(1LL << (HINT_VALUE_SECS_BITS - 1)))
>+#define HINT_VALUE_NSEC_SHIFT (sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS)
>+#define HINT_VALUE_NSEC_MAX ((1ULL << HINT_VALUE_NSEC_BITS) - 1)
>+
> /*
>  * HINT_CLASS_BITS should be big enough to store any mp_class value.
>  * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE.
>@@ -1610,6 +1647,25 @@ hint_uuid_raw(const char *data)
>  return hint_create(MP_CLASS_UUID, val);
> }
> 
>+static inline hint_t
>+hint_datetime(struct datetime *date)
>+{
>+ /*
>+ * Use at most HINT_VALUE_SECS_BITS from datetime
>+ * seconds field as a hint value, and at MSB part
>+ * of HINT_VALUE_NSEC_BITS from nanoseconds.
>+ */
>+ int64_t secs = date->secs;
>+ int32_t nsec = date->nsec;
>+ uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 :
>+ secs - HINT_VALUE_SECS_MIN;
>+ if (val >= HINT_VALUE_SECS_MAX)
>+ val = HINT_VALUE_SECS_MAX;
>+ val <<= HINT_VALUE_NSEC_BITS;
>+ val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX;
>+ return hint_create(MP_CLASS_DATETIME, val);
>+}
>+
> static inline uint64_t
> hint_str_raw(const char *s, uint32_t len)
> {
>@@ -1741,6 +1797,17 @@ field_hint_uuid(const char *field)
>  return hint_uuid_raw(data);
> }
> 
>+static inline hint_t
>+field_hint_datetime(const char *field)
>+{
>+ assert(mp_typeof(*field) == MP_EXT);
>+ int8_t ext_type;
>+ uint32_t len = mp_decode_extl(&field, &ext_type);
>+ assert(ext_type == MP_DATETIME);
>+ struct datetime date;
>+ return hint_datetime(datetime_unpack(&field, len, &date));
>+}
>+
> static inline hint_t
> field_hint_string(const char *field, struct coll *coll)
> {
>@@ -1792,6 +1859,11 @@ field_hint_scalar(const char *field, struct coll *coll)
>  }
>  case MP_UUID:
>  return hint_uuid_raw(field);
>+ case MP_DATETIME:
>+ {
>+ struct datetime date;
>+ return hint_datetime(datetime_unpack(&field, len, &date));
>+ }
>  default:
>  unreachable();
>  }
>@@ -1829,6 +1901,8 @@ field_hint(const char *field, struct coll *coll)
>  return field_hint_decimal(field);
>  case FIELD_TYPE_UUID:
>  return field_hint_uuid(field);
>+ case FIELD_TYPE_DATETIME:
>+ return field_hint_datetime(field);
>  default:
>  unreachable();
>  }
>@@ -1943,6 +2017,9 @@ key_def_set_hint_func(struct key_def *def)
>  case FIELD_TYPE_UUID:
>  key_def_set_hint_func<FIELD_TYPE_UUID>(def);
>  break;
>+ case FIELD_TYPE_DATETIME:
>+ key_def_set_hint_func<FIELD_TYPE_DATETIME>(def);
>+ break;
>  default:
>  /* Invalid key definition. */
>  def->key_hint = NULL;
>diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
>index 7125090e6..ebc05e29c 100644
>--- a/src/lib/core/datetime.c
>+++ b/src/lib/core/datetime.c
>@@ -123,3 +123,12 @@ datetime_to_string(const struct datetime *date, char *buf, int len)
>  return sz;
> }
> 
>+int
>+datetime_compare(const struct datetime *lhs, const struct datetime *rhs)
>+{
>+ int result = COMPARE_RESULT(lhs->secs, rhs->secs);
>+ if (result != 0)
>+ return result;
>+
>+ return COMPARE_RESULT(lhs->nsec, rhs->nsec);
>+}
>diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
>index fb537e372..fc5717115 100644
>--- a/src/lib/core/datetime.h
>+++ b/src/lib/core/datetime.h
>@@ -81,6 +81,17 @@ struct datetime_interval {
>  */
> #define DT_TO_STRING_BUFSIZE 48
> 
>+/*
>+ * Compare arguments of a datetime type
>+ * @param lhs left datetime argument
>+ * @param rhs right datetime argument
>+ * @retval < 0 if lhs less than rhs
>+ * @retval = 0 if lhs and rhs equal
>+ * @retval > 0 if lhs greater than rhs
>+ */
>+int
>+datetime_compare(const struct datetime *lhs, const struct datetime *rhs);
>+
> /**
>  * Convert datetime to string using default format
>  * @param date source datetime value
>diff --git a/test/engine/datetime.result b/test/engine/datetime.result
>new file mode 100644
>index 000000000..848a0aaec
>--- /dev/null
>+++ b/test/engine/datetime.result
>@@ -0,0 +1,77 @@
>+-- test-run result file version 2
>+env = require('test_run')
>+ | ---
>+ | ...
>+test_run = env.new()
>+ | ---
>+ | ...
>+engine = test_run:get_cfg('engine')
>+ | ---
>+ | ...
>+
>+date = require('datetime')
>+ | ---
>+ | ...
>+
>+_ = box.schema.space.create('T', {engine = engine})
>+ | ---
>+ | ...
>+_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
>+ | ---
>+ | ...
>+
>+box.space.T:insert{date('1970-01-01')}\
>+box.space.T:insert{date('1970-01-02')}\
>+box.space.T:insert{date('1970-01-03')}\
>+box.space.T:insert{date('2000-01-01')}
>+ | ---
>+ | ...
>+
>+o = box.space.T:select{}
>+ | ---
>+ | ...
>+assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
>+ | ---
>+ | - true
>+ | ...
>+assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
>+ | ---
>+ | - true
>+ | ...
>+assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
>+ | ---
>+ | - true
>+ | ...
>+assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
>+ | ---
>+ | - true
>+ | ...
>+
>+for i = 1,16 do\
>+ box.space.T:insert{date.now()}\
>+end
>+ | ---
>+ | ...
>+
>+a = box.space.T:select{}
>+ | ---
>+ | ...
>+err = {}
>+ | ---
>+ | ...
>+for i = 1, #a - 1 do\
>+ if a[i][1] >= a[i+1][1] then\
>+ table.insert(err, {a[i][1], a[i+1][1]})\
>+ break\
>+ end\
>+end
>+ | ---
>+ | ...
>+
>+err
>+ | ---
>+ | - []
>+ | ...
>+box.space.T:drop()
>+ | ---
>+ | ...
>diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua
>new file mode 100644
>index 000000000..3685e4d4b
>--- /dev/null
>+++ b/test/engine/datetime.test.lua
>@@ -0,0 +1,35 @@
>+env = require('test_run')
>+test_run = env.new()
>+engine = test_run:get_cfg('engine')
>+
>+date = require('datetime')
>+
>+_ = box.schema.space.create('T', {engine = engine})
>+_ = box.space.T:create_index('pk', {parts={1,'datetime'}})
>+
>+box.space.T:insert{date('1970-01-01')}\
>+box.space.T:insert{date('1970-01-02')}\
>+box.space.T:insert{date('1970-01-03')}\
>+box.space.T:insert{date('2000-01-01')}
>+
>+o = box.space.T:select{}
>+assert(tostring(o[1][1]) == '1970-01-01T00:00Z')
>+assert(tostring(o[2][1]) == '1970-01-02T00:00Z')
>+assert(tostring(o[3][1]) == '1970-01-03T00:00Z')
>+assert(tostring(o[4][1]) == '2000-01-01T00:00Z')
>+
>+for i = 1,16 do\
>+ box.space.T:insert{date.now()}\
>+end
>+
>+a = box.space.T:select{}
>+err = {}
>+for i = 1, #a - 1 do\
>+ if a[i][1] >= a[i+1][1] then\
>+ table.insert(err, {a[i][1], a[i+1][1]})\
>+ break\
>+ end\
>+end
>+
>+err
>+box.space.T:drop()
>diff --git a/test/unit/datetime.c b/test/unit/datetime.c
>index c6ba444f5..61ebfe76b 100644
>--- a/test/unit/datetime.c
>+++ b/test/unit/datetime.c
>@@ -281,7 +281,7 @@ mp_datetime_test()
>  };
>  size_t index;
> 
>- plan(68);
>+ plan(85);
>  for (index = 0; index < lengthof(tests); index++) {
>  struct datetime date = {
>  tests[index].secs,
>@@ -303,6 +303,7 @@ mp_datetime_test()
>  struct datetime *rc = mp_decode_datetime(&data1, &ret);
>  is(rc, &ret, "mp_decode_datetime() return code");
>  is(data1, end, "mp_sizeof_uuid() == decoded length");
>+ is(datetime_compare(&date, &ret), 0, "datetime_compare(&date, &ret)");
>  }
>  check_plan();
> }
>--
>2.29.2 
 
 
--
Dmitry E. Oboukhov
 

[-- Attachment #2: Type: text/html, Size: 21933 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices
  2021-08-19 11:18   ` UNera via Tarantool-patches
@ 2021-08-19 11:53     ` Safin Timur via Tarantool-patches
  2021-08-19 14:47       ` Dmitry E. Oboukhov via Tarantool-patches
  0 siblings, 1 reply; 20+ messages in thread
From: Safin Timur via Tarantool-patches @ 2021-08-19 11:53 UTC (permalink / raw)
  To: unera; +Cc: tarantool-patches, v.shpilevoy



On 19.08.2021 14:18, unera@tarantool.org wrote:
> Hi!
> I looked trhough the patchset.
> I also compiled the bracnch and tested by hand.
> Example:
> tarantool> datetime = require('datetime')
> ---
> ...
> tarantool> datetime.new('2020-01-02 02:00')
> ---
> - 1970-01-01T00:00Z
> ...
> tarantool> datetime.new('2020-01-02 02:00+0300')
> ---
> - 1970-01-01T00:00Z
> ...
> tarantool> datetime.new('2020-01-02 02:00:11+0300')
> ---
> - 1970-01-01T00:00Z
> ...
> tarantool> datetime.new('2020-01-02T02:00:11+0300')
> ---
> - 1970-01-01T00:00Z
> ...
> The code looks like as broken.

Not exactly :)

.new{} is constructor which is not expecting string at the moment. 
.parse() expect strings, or defaul __call handler for the module, which 
is dispatching between parsing, if passed string, and, direct calling 
constructor, if there are no arguments passed or they are object.

```
tarantool> date('2020-01-02 02:00')
---
- 2020-01-02T02:00Z
...

tarantool> date('2020-01-02 02:00')
---
- 2020-01-02T02:00Z
...

tarantool> date('2020-01-02 02:00:11+0300')
---
- 2020-01-02T02:00:11+03:00
...

tarantool> date('2020-01-02T02:00:11+0300')
---
- 2020-01-02T02:00:11+03:00
...
```

> Also my notes:
> 
>  1. tostring has to return datetime string with seconds and timezone

Will add required seconds - they are reduced now for more compact form 
like here below.
------------------------------------------------
	if (second || nanosec) {
		SNPRINT(sz, snprintf, buf, len, ":%02d", second);
		if (nanosec) {
------------------------------------------------

here is incrementa change which enforce required seconds:
------------------------------------------------
diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
index ebc05e29c..6e2a76da5 100644
--- a/src/lib/core/datetime.c
+++ b/src/lib/core/datetime.c
@@ -93,20 +93,17 @@ datetime_to_string(const struct datetime *date, char 
*buf, int len)
  	nanosec = date->nsec;

  	int sz = 0;
-	SNPRINT(sz, snprintf, buf, len, "%04d-%02d-%02dT%02d:%02d",
-		year, month, day, hour, minute);
-	if (second || nanosec) {
-		SNPRINT(sz, snprintf, buf, len, ":%02d", second);
-		if (nanosec) {
-			if ((nanosec % 1000000) == 0)
-				SNPRINT(sz, snprintf, buf, len, ".%03d",
-					nanosec / 1000000);
-			else if ((nanosec % 1000) == 0)
-				SNPRINT(sz, snprintf, buf, len, ".%06d",
-					nanosec / 1000);
-			else
-				SNPRINT(sz, snprintf, buf, len, ".%09d", nanosec);
-		}
+	SNPRINT(sz, snprintf, buf, len, "%04d-%02d-%02dT%02d:%02d:%02d",
+		year, month, day, hour, minute, second);
+	if (nanosec != 0) {
+		if ((nanosec % 1000000) == 0)
+			SNPRINT(sz, snprintf, buf, len, ".%03d",
+				nanosec / 1000000);
+		else if ((nanosec % 1000) == 0)
+			SNPRINT(sz, snprintf, buf, len, ".%06d",
+				nanosec / 1000);
+		else
+			SNPRINT(sz, snprintf, buf, len, ".%09d", nanosec);
  	}
  	if (offset == 0) {
  		SNPRINT(sz, snprintf, buf, len, "Z");
------------------------------------------------
>  2. also the other databases try to avoid to use «Z» symbol in
>     datetime-format, so I thing we should use «+00:00» instead «Z»

I don't care which way to output UTC, but here I need more examples 
(which vendor, how and when outputs timezone)...

Like - here is DATE WITH TIMEZONE column, and here is DATE WITHOUT 
TIMEZONE. And here TIMESTAMP WITHOUT TIMEZONE, and with UTC, and we see...

> 
> Yesterday we approved the following API:
> T:add{year=XXX, month=YYY, ...}
> T:sub{year=XXX, month=YYY, ...}
> T:set{year=XXX, …, tz=’+03:00’}
> T:strftime(‘%F %D %z’)
> I want to see a test set for the API.

FWIW, :add{}/:sub{}/:strftime{} are there, but not :set() as 
unimplemented yet.

> 
> Also we approved the following methods:
> T:sec() — 35
> T:min() — 12
> T:day() — 19
> T:wday() — 5
> T:yday() — 231
> T:year() — 2021
> T:month() — 8
> T:hour() — 23
> T:tz() — ‘+03:00’

Those have not yet been exported, yes.

And BTW, tz() assumed to return numeric information (difference in 
minutes to UTC), not string representation?

What about :totable() - I've got an impression that we also approved it. 
Yes?

> It would be nice to see test set for each of the functions.
> Also it would be nice to rename ambigouos items «secs» to «epoch» or 
> «timestamp»

Here I am confused - which "items" do you mean here? [Internal strcuture 
fields, or some API elements?]

> Datetime object must have methods to return integer and float timestamp:
> T:epoch()
> T:timestamp()

epoch() is integer, and timestamp() is double, correct?

> 
>     Четверг, 19 августа 2021, 5:58 +03:00 от Timur Safin via
>     Tarantool-patches <tarantool-patches@dev.tarantool.org>:
>     * storage hints implemented for datetime_t values;
>     * proper comparison for indices of datetime type.
> 
>     Part of #5941
>     Part of #5946
> 
>     @TarantoolBot document
> 
>     Title: Storage support for datetime values
> 
>     It's now possible to store datetime values in spaces and create
>     indexed datetime fields.
> 
>     Please refer to
>     https://github.com/tarantool/tarantool/discussions/6244
>     <https://github.com/tarantool/tarantool/discussions/6244>
>     for more detailed description of a storage schema.
> 
> --
> Dmitry E. Oboukhov

Thanks,
Timur


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices
  2021-08-19 11:53     ` Safin Timur via Tarantool-patches
@ 2021-08-19 14:47       ` Dmitry E. Oboukhov via Tarantool-patches
  0 siblings, 0 replies; 20+ messages in thread
From: Dmitry E. Oboukhov via Tarantool-patches @ 2021-08-19 14:47 UTC (permalink / raw)
  To: Safin Timur; +Cc: tarantool-patches, v.shpilevoy

> > The code looks like as broken.
> 
> Not exactly :)
> 
> .new{} is constructor which is not expecting string at the moment. .parse()
> expect strings, or defaul __call handler for the module, which is
> dispatching between parsing, if passed string, and, direct calling
> constructor, if there are no arguments passed or they are object.

If .new and .__call are different, .new have to throw an exception
instead creating object with zero in its timestamp.

I think it would be nice to drop '__call' method and patch 'new' for
work with strings (mainstream) and tables (rare cases)

> diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
> index ebc05e29c..6e2a76da5 100644
> --- a/src/lib/core/datetime.c
> +++ b/src/lib/core/datetime.c
> @@ -93,20 +93,17 @@ datetime_to_string(const struct datetime *date, char

I can't apply the patch:

> patching file src/lib/core/datetime.c
> patch: **** malformed patch at line 4: *buf, int len)

I think it isn't between previous and current versions.

> ------------------------------------------------
> >  2. also the other databases try to avoid to use «Z» symbol in
> >     datetime-format, so I thing we should use «+00:00» instead «Z»
> 
> I don't care which way to output UTC, but here I need more examples (which
> vendor, how and when outputs timezone)...
> 
> Like - here is DATE WITH TIMEZONE column, and here is DATE WITHOUT TIMEZONE.
> And here TIMESTAMP WITHOUT TIMEZONE, and with UTC, and we see...

Example Postgresql

unera=> SET TIME ZONE 'ZULU';
SET
unera=> select '2021-08-19 17:36:14+0300'::TIMESTAMPTZ(0);
      timestamptz       
------------------------
 2021-08-19 14:36:14+00
(1 строка)

unera=> select '2021-08-19 17:36:14Z'::TIMESTAMPTZ(0);
      timestamptz       
------------------------
 2021-08-19 17:36:14+00
(1 строка)

Even if you set current timezone to 'Z' it prints zone as number.
So user can sort these strings as strings and sort will be correct.


> 
> > 
> > Yesterday we approved the following API:
> > T:add{year=XXX, month=YYY, ...}
> > T:sub{year=XXX, month=YYY, ...}
> > T:set{year=XXX, …, tz=’+03:00’}
> > T:strftime(‘%F %D %z’)
> > I want to see a test set for the API.
> 
> FWIW, :add{}/:sub{}/:strftime{} are there, but not :set() as unimplemented
> yet.

Ok, I think we need to have test for each method.
 
> > Datetime object must have methods to return integer and float timestamp:
> > T:epoch()
> > T:timestamp()
> 
> epoch() is integer, and timestamp() is double, correct?
ok

-- 

. ''`.            Dmitry E. Oboukhov <unera@debian.org>
: :’  :                           <unera@tarantool.org>
`. `~’                  work: <d.oboukhov@corp.mail.ru>
  `- 71ED ACFC 6801 0DD9 1AD1  9B86 8D1F 969A 08EE A756

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime
  2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime Timur Safin via Tarantool-patches
  2021-08-19  9:43   ` Serge Petrenko via Tarantool-patches
@ 2021-08-19 15:26   ` Vladimir Davydov via Tarantool-patches
  2021-08-24 21:13     ` Vladislav Shpilevoy via Tarantool-patches
  1 sibling, 1 reply; 20+ messages in thread
From: Vladimir Davydov via Tarantool-patches @ 2021-08-19 15:26 UTC (permalink / raw)
  To: Timur Safin; +Cc: tarantool-patches, v.shpilevoy

On Thu, Aug 19, 2021 at 05:56:29AM +0300, Timur Safin wrote:
> New third_party module - c-dt
> -----------------------------
> 
> * Integrated chansen/c-dt parser as 3rd party module to the
>   Tarantool cmake build process;
> * Points to tarantool/c-dt instead of original chansen/c-dt to
>   have easier build integration, because there are additional
>   commits which have integrated cmake support and have established
>   symbols renaming facilities (similar to those we see in xxhash
>   or icu).
> * took special care that generated build artifacts not override
>   in-source files, but use different build/ directory.
> 
> * added datetime parsing unit tests:
>   - for literals - with and without trailing timezones;
>   - we check that strftime is reversible and produce consistent
>     results after roundtrip from/to strings;
> 
> New built-in module `datetime`
> ------------------------------
> 
> * created a new Tarantool built-in module `datetime`;
> * register new cdef types for this module;
> * reexported some `dt_*` functions from `c-dt` library.
>   We have to rename externally public symbols to avoid
>   name clashes possible usage of this same party module but
>   from inside of dynamic libraries. We prefix c-dt symbols
>   in the Tarantool build with `tnt_` prefix;
> 
> * `strftime` implemented as a simple ffi wrappers in C
>   kernel code:
> 
> We used to use heavy ffi-related Lua code, but it appears
> to be quite fragile, if we run this code on different OS
> targets. To make this Lua code less platfrom dependent we
> have moved all platform specifics to the C level.
> Specifically we want to avoid dependence on `struct tm {}`
> layout and GLIBC-specific `strftime` behaviour (which
> differs to BSD/Mac LIBC behaviour).
> 
> * display datetime
> 
>   - introduced output routine for converting datetime
>     to their default output format.
> 
>   - use this routine for tostring() in datetime.lua
> 
> - verify that we parse expected, and fail with unexpected:
>   - for that, in the datetime.lua, we have extended api of `parse_date()`,
>     `parse_time()`, and `parse_time_zone()` so they return
>     not only parsed object, but also a length of parsed substring;
>   - which allows us to parse even _partially_ valid strings like
>     "20121224 Foo bar".
> 
> * simplified interfaces
>   - totable() export table values similar to os.date('*t')
>   - set() provide unified interface to set values using
>     the same set of attributes as in totable()
> 
> Part of #5941
> 
> @TarantoolBot document
> Title: Introduced new built-in `datetime` module
> 
> `datetime` module has been introduced, which allows to parse
> ISO-8601 literals representing timestamps of various formats,
> and then manipulate with date objects.
> 
> Please refer to https://github.com/tarantool/tarantool/discussions/6244
> for more detailed description of module API.
> 
> Part of #5941
> ---
>  .gitmodules                    |   3 +
>  CMakeLists.txt                 |   8 +
>  cmake/BuildCDT.cmake           |  10 +
>  extra/exports                  |  32 ++
>  src/CMakeLists.txt             |   5 +-
>  src/lib/core/CMakeLists.txt    |   1 +
>  src/lib/core/datetime.c        | 125 ++++++++
>  src/lib/core/datetime.h        |  86 +++++
>  src/lua/datetime.lua           | 567 +++++++++++++++++++++++++++++++++
>  src/lua/init.c                 |   4 +-
>  src/lua/utils.c                |  27 ++
>  src/lua/utils.h                |  12 +
>  test/app-tap/datetime.test.lua | 213 +++++++++++++

There must be many more Lua tests covering all possible cases.
Currently, the module doesn't seem to work quite as expected:

	tarantool> datetime.new({year = 2000})
	---
	- 1999-11-30T00:00Z
	...

>  test/unit/CMakeLists.txt       |   3 +-
>  test/unit/datetime.c           | 260 +++++++++++++++
>  test/unit/datetime.result      | 358 +++++++++++++++++++++
>  third_party/c-dt               |   1 +
>  17 files changed, 1712 insertions(+), 3 deletions(-)
>  create mode 100644 cmake/BuildCDT.cmake
>  create mode 100644 src/lib/core/datetime.c
>  create mode 100644 src/lib/core/datetime.h
>  create mode 100644 src/lua/datetime.lua
>  create mode 100755 test/app-tap/datetime.test.lua
>  create mode 100644 test/unit/datetime.c
>  create mode 100644 test/unit/datetime.result
>  create mode 160000 third_party/c-dt
> 
> diff --git a/.gitmodules b/.gitmodules
> index f2f91ee72..aa3fbae4e 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -43,3 +43,6 @@
>  [submodule "third_party/xxHash"]
>  	path = third_party/xxHash
>  	url = https://github.com/tarantool/xxHash
> +[submodule "third_party/c-dt"]
> +	path = third_party/c-dt
> +	url = https://github.com/tarantool/c-dt.git
> diff --git a/CMakeLists.txt b/CMakeLists.txt
> index e25b81eac..8037c30a7 100644
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -571,6 +571,14 @@ endif()
>  # zstd
>  #
>  
> +#
> +# Christian Hansen c-dt
> +#
> +
> +include(BuildCDT)
> +libccdt_build()
> +add_dependencies(build_bundled_libs cdt)
> +
>  #
>  # Third-Party misc
>  #
> diff --git a/cmake/BuildCDT.cmake b/cmake/BuildCDT.cmake
> new file mode 100644
> index 000000000..80b26c64a
> --- /dev/null
> +++ b/cmake/BuildCDT.cmake
> @@ -0,0 +1,10 @@
> +macro(libccdt_build)
> +    set(LIBCDT_INCLUDE_DIRS ${PROJECT_SOURCE_DIR}/third_party/c-dt/)
> +    set(LIBCDT_LIBRARIES cdt)
> +
> +    file(MAKE_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt/build/)
> +    add_subdirectory(${PROJECT_SOURCE_DIR}/third_party/c-dt
> +                     ${CMAKE_CURRENT_BINARY_DIR}/third_party/c-dt/build/)
> +    set_target_properties(cdt PROPERTIES COMPILE_FLAGS "-DDT_NAMESPACE=tnt_")
> +    add_definitions("-DDT_NAMESPACE=tnt_")
> +endmacro()
> diff --git a/extra/exports b/extra/exports
> index 9eaba1282..0e7392d61 100644
> --- a/extra/exports
> +++ b/extra/exports
> @@ -148,8 +148,16 @@ csv_feed
>  csv_iterator_create
>  csv_next
>  csv_setopt
> +datetime_now
> +datetime_strftime
> +datetime_to_string
> +datetime_unpack
>  decimal_from_string
>  decimal_unpack
> +dt_year
> +dt_month
> +dt_doy
> +dt_dom

Why no tnt_ prefix?

>  error_ref
>  error_set_prev
>  error_unref
> @@ -447,6 +455,30 @@ title_set_status
>  title_update
>  tnt_default_cert_dir_paths
>  tnt_default_cert_file_paths
> +tnt_dt_add_years
> +tnt_dt_add_quarters
> +tnt_dt_add_months
> +tnt_dt_dow
> +tnt_dt_from_rdn
> +tnt_dt_from_struct_tm

This function is unused, and so are a few others below.
Should remove them from this list?

> +tnt_dt_from_yd
> +tnt_dt_from_ymd
> +tnt_dt_from_yqd
> +tnt_dt_from_ywd
> +tnt_dt_parse_iso_date
> +tnt_dt_parse_iso_time_basic
> +tnt_dt_parse_iso_time_extended
> +tnt_dt_parse_iso_time
> +tnt_dt_parse_iso_zone_basic
> +tnt_dt_parse_iso_zone_extended
> +tnt_dt_parse_iso_zone_lenient
> +tnt_dt_parse_iso_zone
> +tnt_dt_rdn
> +tnt_dt_to_struct_tm
> +tnt_dt_to_yd
> +tnt_dt_to_ymd
> +tnt_dt_to_yqd
> +tnt_dt_to_ywd
>  tnt_iconv
>  tnt_iconv_close
>  tnt_iconv_open
> diff --git a/src/lib/core/datetime.c b/src/lib/core/datetime.c
> new file mode 100644
> index 000000000..7125090e6
> --- /dev/null
> +++ b/src/lib/core/datetime.c
> @@ -0,0 +1,125 @@
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + */
> +
> +#include <assert.h>
> +#include <limits.h>
> +#include <string.h>
> +#include <time.h>
> +
> +#include "trivia/util.h"
> +#include "datetime.h"
> +
> +/*

Should be /**. It's not about doxygen, it's in accordance with our
coding style:

https://www.tarantool.io/en/doc/latest/dev_guide/c_style_guide/#chapter-8-commenting

(BTW parenthesis after sizeof are also mandatory)

Please fix here and everywhere else.

> + * Given the seconds from Epoch (1970-01-01) we calculate date
> + * since Rata Die (0001-01-01).
> + * DT_EPOCH_1970_OFFSET is the distance in days from Rata Die to Epoch.
> + */
> +static int
> +local_dt(int64_t secs)
> +{
> +	return dt_from_rdn((int)(secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET);
> +}
> +
> +static struct tm *
> +datetime_to_tm(const struct datetime *date)
> +{
> +	static struct tm tm;

There's no reason to use a static variable here. Please pass tm by
pointer.

> +
> +	memset(&tm, 0, sizeof(tm));
> +	int64_t secs = date->secs;
> +	dt_to_struct_tm(local_dt(secs), &tm);
> +
> +	int seconds_of_day = (int64_t)date->secs % SECS_PER_DAY;
> +	tm.tm_hour = (seconds_of_day / 3600) % 24;
> +	tm.tm_min = (seconds_of_day / 60) % 60;
> +	tm.tm_sec = seconds_of_day % 60;

Please use constants here as well: HOURS_PER_DAY, SECS_PER_MINUTE, etc.

Please replace here and everywhere else.

> +
> +	return &tm;
> +}
> +
> +void
> +datetime_now(struct datetime *now)
> +{
> +	struct timeval tv;
> +	gettimeofday(&tv, NULL);
> +	now->secs = tv.tv_sec;
> +	now->nsec = tv.tv_usec * 1000;
> +
> +	time_t now_seconds;
> +	time(&now_seconds);
> +	struct tm tm;
> +	localtime_r(&now_seconds, &tm);

Please either implement local timezone caching (preferrably) or add a
todo:

/*
 * TODO(gh-XXXX): ...
 */

gh-XXXX is the number of GitHub ticket to track the work.

> +	now->offset = tm.tm_gmtoff / 60;
> +}
> +
> +size_t
> +datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +		  uint32_t len)

I would reorder the arguments for consistency with strftime and
compatibility with SNPRINT:

datetime_strftime(buf, len, fmt, date);

> +{
> +	struct tm *p_tm = datetime_to_tm(date);
> +	return strftime(buf, len, fmt, p_tm);
> +}
> +
> +
> +/* NB! buf may be NULL, and we should handle it gracefully, returning
> + * calculated length of output string
> + */
> +int
> +datetime_to_string(const struct datetime *date, char *buf, int len)

datetime_snprint(buf, len, date);

for compatibility with SNPRINT and consistency with our other
snprint-like functions.

> +{
> +	int offset = date->offset;
> +	/* for negative offsets around Epoch date we could get
> +	 * negative secs value, which should be attributed to
> +	 * 1969-12-31, not 1970-01-01, thus we first shift
> +	 * epoch to Rata Die then divide by seconds per day,
> +	 * not in reverse
> +	 */
> +	int64_t rd_seconds = (int64_t)date->secs + offset * 60 +
> +			     SECS_EPOCH_1970_OFFSET;
> +	int rd_number = rd_seconds / SECS_PER_DAY;
> +	assert(rd_number <= INT_MAX);
> +	assert(rd_number >= INT_MIN);
> +	dt_t dt = dt_from_rdn(rd_number);
> +
> +	int year, month, day, second, nanosec, sign;
> +	dt_to_ymd(dt, &year, &month, &day);
> +
> +	int hour = (rd_seconds / 3600) % 24;
> +	int minute = (rd_seconds / 60) % 60;
> +	second = rd_seconds % 60;
> +	nanosec = date->nsec;
> +
> +	int sz = 0;
> +	SNPRINT(sz, snprintf, buf, len, "%04d-%02d-%02dT%02d:%02d",
> +		year, month, day, hour, minute);
> +	if (second || nanosec) {
> +		SNPRINT(sz, snprintf, buf, len, ":%02d", second);
> +		if (nanosec) {
> +			if ((nanosec % 1000000) == 0)
> +				SNPRINT(sz, snprintf, buf, len, ".%03d",
> +					nanosec / 1000000);
> +			else if ((nanosec % 1000) == 0)
> +				SNPRINT(sz, snprintf, buf, len, ".%06d",
> +					nanosec / 1000);
> +			else
> +				SNPRINT(sz, snprintf, buf, len, ".%09d", nanosec);
> +		}
> +	}
> +	if (offset == 0) {
> +		SNPRINT(sz, snprintf, buf, len, "Z");
> +	} else {
> +		if (offset < 0) {
> +			sign = '-';
> +			offset = -offset;
> +		} else {
> +			sign = '+';
> +		}
> +		SNPRINT(sz, snprintf, buf, len, "%c%02d:%02d", sign,
> +			offset / 60, offset % 60);
> +	}
> +	return sz;
> +}
> +
> diff --git a/src/lib/core/datetime.h b/src/lib/core/datetime.h
> new file mode 100644
> index 000000000..71feefded
> --- /dev/null
> +++ b/src/lib/core/datetime.h
> @@ -0,0 +1,86 @@
> +#pragma once
> +/*
> + * SPDX-License-Identifier: BSD-2-Clause
> + *
> + * Copyright 2021, Tarantool AUTHORS, please see AUTHORS file.
> + */
> +
> +#include <stdint.h>
> +#include <stdbool.h>
> +#include "c-dt/dt.h"
> +
> +#if defined(__cplusplus)
> +extern "C"
> +{
> +#endif /* defined(__cplusplus) */
> +
> +/**
> + * We count dates since so called "Rata Die" date
> + * January 1, 0001, Monday (as Day 1).
> + * But datetime structure keeps seconds since
> + * Unix "Epoch" date:
> + * Unix, January 1, 1970, Thursday
> + *
> + * The difference between Epoch (1970-01-01)
> + * and Rata Die (0001-01-01) is 719163 days.
> + */
> +
> +#ifndef SECS_PER_DAY
> +#define SECS_PER_DAY          86400
> +#define DT_EPOCH_1970_OFFSET  719163
> +#endif
> +
> +#define SECS_EPOCH_1970_OFFSET 	\
> +	((int64_t)DT_EPOCH_1970_OFFSET * SECS_PER_DAY)

DT_EPOCH_1970_OFFSET
SECS_EPOCH_1970_OFFSET

Inconsistent naming. Looks like DT_ is a namespace-like prefix.
I suggest

EPOCH_1970_OFFSET_DAYS
EPOCH_1970_OFFSET_SECS

Maybe with DT_ prefix, although I don't think it's necessary, because
these seem like universal constants.

> +/**
> + * datetime structure keeps number of seconds since
> + * Unix Epoch.
> + * Time is normalized by UTC, so time-zone offset
> + * is informative only.
> + */
> +struct datetime {
> +	/** Seconds since Epoch. */
> +	double secs;

epoch

Please add a comment explaining why you use double here.

> +	/** Nanoseconds, if any. */
> +	uint32_t nsec;

AFAIK unsigned types are typically less effecient than signed, because
the compiler has to correctly handle overflow for them. That's why it is
often discouraged to use an unsigned type even if the variable never
stores a value < 0.

> +	/** Offset in minutes from UTC. */
> +	int32_t offset;

tzoffset

> +};
> +
> +/**
> + * Date/time interval structure
> + */
> +struct datetime_interval {
> +	/** Relative seconds delta. */
> +	double secs;
> +	/** Nanoseconds delta, if any. */
> +	uint32_t nsec;
> +};

Not used. Please remove.

> +
> +/**
> + * Convert datetime to string using default format
> + * @param date source datetime value
> + * @param buf output character buffer
> + * @param len size ofoutput buffer
> + */
> +int
> +datetime_to_string(const struct datetime *date, char *buf, int len);
> +
> +/**
> + * Convert datetime to string using default format provided
> + * Wrapper around standard strftime() function
> + * @param date source datetime value
> + * @param fmt format
> + * @param buf output buffer
> + * @param len size of output buffer
> + */
> +size_t
> +datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +		  uint32_t len);
> +
> +void
> +datetime_now(struct datetime *now);

Missing comment. Please add.

> +
> +#if defined(__cplusplus)
> +} /* extern "C" */
> +#endif /* defined(__cplusplus) */
> diff --git a/src/lua/datetime.lua b/src/lua/datetime.lua
> new file mode 100644
> index 000000000..88aad0744
> --- /dev/null
> +++ b/src/lua/datetime.lua
> @@ -0,0 +1,567 @@
> +local ffi = require('ffi')
> +
> +--[[
> +    `c-dt` library functions handles properly both positive and negative `dt`
> +    values, where `dt` is a number of dates since Rata Die date (0001-01-01).
> +
> +    For better compactness of our typical data in MessagePack stream we shift
> +    root of our time to the Unix Epoch date (1970-01-01), thus our 0 is
> +    actually dt = 719163.
> +
> +    So here is a simple formula how convert our epoch-based seconds to dt values
> +        dt = (secs / 86400) + 719163
> +    Where 719163 is an offset of Unix Epoch (1970-01-01) since Rata Die
> +    (0001-01-01) in dates.
> +]]
> +
> +-- dt_core.h definitions
> +ffi.cdef [[
> +
> +typedef int dt_t;
> +
> +typedef enum {
> +    DT_MON       = 1,
> +    DT_MONDAY    = 1,
> +    DT_TUE       = 2,
> +    DT_TUESDAY   = 2,
> +    DT_WED       = 3,
> +    DT_WEDNESDAY = 3,
> +    DT_THU       = 4,
> +    DT_THURSDAY  = 4,
> +    DT_FRI       = 5,
> +    DT_FRIDAY    = 5,
> +    DT_SAT       = 6,
> +    DT_SATURDAY  = 6,
> +    DT_SUN       = 7,
> +    DT_SUNDAY    = 7,
> +} dt_dow_t;
> +
> +dt_t   tnt_dt_from_rdn     (int n);
> +dt_t   tnt_dt_from_ymd     (int y, int m, int d);
> +int    tnt_dt_rdn          (dt_t dt);
> +
> +dt_dow_t tnt_dt_dow        (dt_t dt);
> +
> +]]
> +
> +-- dt_accessor.h
> +ffi.cdef [[
> +
> +int     dt_year         (dt_t dt);
> +int     dt_month        (dt_t dt);
> +
> +int     dt_doy          (dt_t dt);
> +int     dt_dom          (dt_t dt);
> +
> +]]
> +
> +-- dt_arithmetic.h definitions
> +ffi.cdef [[
> +
> +typedef enum {
> +    DT_EXCESS,
> +    DT_LIMIT,
> +    DT_SNAP
> +} dt_adjust_t;
> +
> +dt_t   tnt_dt_add_years    (dt_t dt, int delta, dt_adjust_t adjust);
> +dt_t   tnt_dt_add_quarters (dt_t dt, int delta, dt_adjust_t adjust);
> +dt_t   tnt_dt_add_months   (dt_t dt, int delta, dt_adjust_t adjust);
> +
> +]]
> +
> +-- dt_parse_iso.h definitions
> +ffi.cdef [[
> +
> +size_t tnt_dt_parse_iso_date (const char *str, size_t len, dt_t *dt);
> +size_t tnt_dt_parse_iso_time (const char *str, size_t len, int *sod, int *nsec);
> +size_t tnt_dt_parse_iso_zone_lenient(const char *str, size_t len, int *offset);

Why not hide all dt internals behind C wrappers defined in the datetime
module? This would make the Lua code easier for understanding, because
e.g. 'parse' would call just one C functions instead of three. Also,
this would expose useful datetime helpers (like adding years to a date
or parsing from string to C code), which could be useful. We wouldn't
have to append tnt_ prefix or export any dt functions then.

> +
> +]]
> +
> +-- Tarantool functions - datetime.c
> +ffi.cdef [[
> +
> +int    datetime_to_string(const struct datetime * date, char *buf, int len);

Extra space after '*'. Please self-check coding style.

> +size_t datetime_strftime(const struct datetime *date, const char *fmt, char *buf,
> +                         uint32_t len);
> +void   datetime_now(struct datetime *now);
> +
> +]]

I don't see any point in splitting cdef. I'd put everything in one cdef,
like it was before.

> +
> +local builtin = ffi.C
> +local math_modf = math.modf
> +local math_floor = math.floor
> +
> +local SECS_PER_DAY     = 86400
> +local NANOS_PER_SEC    = 1000000000

NSECS_PER_SEC seems to be a more conventional name.

> +
> +-- c-dt/dt_config.h
> +
> +-- Unix, January 1, 1970, Thursday
> +local DT_EPOCH_1970_OFFSET = 719163
> +
> +
> +local datetime_t = ffi.typeof('struct datetime')
> +
> +local function is_datetime(o)
> +    return ffi.istype(datetime_t, o)
> +end
> +
> +local function check_date(o, message)
> +    if not is_datetime(o) then
> +        return error(("%s: expected datetime, but received %s"):
> +                     format(message, o), 2)
> +    end
> +end
> +
> +local function check_str(s, message)
> +    if not type(s) == 'string' then
> +        return error(("%s: expected string, but received %s"):
> +                     format(message, s), 2)
> +    end
> +end
> +
> +local function check_range(v, range, txt)
> +    assert(#range == 2)
> +    if v < range[1] or v > range[2] then
> +        error(('value %d of %s is out of allowed range [%d, %d]'):
> +              format(v, txt, range[1], range[2]), 4)

I think it would be more effecient to pass range[1] and range[2] as
arguments: check_range(val, min, max, name), because you wouldn't create
a table per each call then.

> +    end
> +end
> +
> +local SECS_EPOCH_OFFSET = (DT_EPOCH_1970_OFFSET * SECS_PER_DAY)
> +
> +local function local_rd(secs)
> +    return math_floor((secs + SECS_EPOCH_OFFSET) / SECS_PER_DAY)
> +end
> +
> +local function local_dt(secs)
> +    return builtin.tnt_dt_from_rdn(local_rd(secs))
> +end

The names of these functions are really unfortunate - I keep forgetting
what they do. There should be comments. This is another reason to hide
everything behind the datetime library - then we wouldn't need to expose
them outside datetime.c.

> +
> +local function normalize_nsec(secs, nsec)

secs, but nsec - inconsistent :-(

Would be nice to use the same names throughout the code.

> +    if nsec < 0 then
> +        secs = secs - 1
> +        nsec = nsec + NANOS_PER_SEC
> +    elseif nsec >= NANOS_PER_SEC then
> +        secs = secs + 1
> +        nsec = nsec - NANOS_PER_SEC
> +    end
> +    return secs, nsec
> +end
> +
> +local function datetime_cmp(lhs, rhs)
> +    if not is_datetime(lhs) or not is_datetime(rhs) then
> +        return nil
> +    end
> +    local sdiff = lhs.secs - rhs.secs
> +    return sdiff ~= 0 and sdiff or (lhs.nsec - rhs.nsec)
> +end
> +
> +local function datetime_eq(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc ~= nil and rc == 0
> +end
> +
> +local function datetime_lt(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc == nil and error('incompatible types for comparison', 2) or
> +           rc < 0
> +end
> +
> +local function datetime_le(lhs, rhs)
> +    local rc = datetime_cmp(lhs, rhs)
> +    return rc == nil and error('incompatible types for comparison', 2) or
> +           rc <= 0
> +end
> +
> +local function datetime_serialize(self)
> +    return { secs = self.secs, nsec = self.nsec, offset = self.offset }
> +end

I don't think this should be visibile to the user, even via __serialize.
I think __serialize should return the same value as __tostring.

> +
> +local parse_zone

Please avoid forward declarations unless absolutely necessary.
Better move the function definition upper.

> +
> +local function datetime_new_raw(secs, nsec, offset)
> +    local dt_obj = ffi.new(datetime_t)
> +    dt_obj.secs = secs
> +    dt_obj.nsec = nsec
> +    dt_obj.offset = offset
> +    return dt_obj
> +end
> +
> +local function datetime_new_dt(dt, secs, fraction, offset)
> +    local epochV = dt ~= nil and (builtin.tnt_dt_rdn(dt) - DT_EPOCH_1970_OFFSET) *
> +                   SECS_PER_DAY or 0
> +    local secsV = secs or 0
> +    local fracV = fraction or 0
> +    local ofsV = offset or 0

Bad variable naming - violates our coding style (we use underscores for
local variable names, not camel-case).

I wouldn't even bother defining new local variables - we can overwrite
the argument values:

    secs = secs or 0
    fraction = fraction or 0
    offset = offset or 0

> +    return datetime_new_raw(epochV + secsV - ofsV * 60, fracV, ofsV)
> +end
> +
> +-- create datetime given attribute values from obj
> +-- { secs = N, nsec = M, offset = O}
> +local function datetime_new_obj(obj, ...)
> +    if obj == nil or type(obj) == 'table' then
> +        return ffi.new(datetime_t, obj)
> +    else
> +        return datetime_new_raw(obj, ...)
> +    end
> +end
> +
> +-- create datetime given attribute values from obj

Please add a comment what obj is. If it's a table, what fields are
expected and what happens if a field is missing.

> +local function datetime_new(obj)
> +    if obj == nil or type(obj) ~= 'table' then
> +        return datetime_new_raw(0, 0, 0)
> +    end

There should be three variants of datetime.new:

  datetime.new()
  datetime.new(str)
  datetime.new(table)

everything else should raise an error.

> +    local ymd = false
> +
> +    local nsec = 0
> +    local hms = false
> +
> +    local dt = 0
> +

The way you place empty lines within functions seems random to me.
Better not use them at all.

> +    local y = obj.year

I don't see much point in all these local variables (y, M, h, s, etc) -
they only clutter the code. Should be as short and simple as:

local function datetime_new(o)
    ...
    check_range(o.year, 1, 9999, 'year')
    check_range(o.month, 1, 12, 'month')
    ...
    if o.year or o.month or o.day then
        ...
    end
    if o.hour or o.min or o.sec then
        ...
    end
    ...
end

> +    if y ~= nil then
> +        check_range(y, {1, 9999}, 'year')
> +        ymd = true

datetime.new{year = 0} doesn't work, but datetime.new{} works and
creates a date with year equal to 0...

> +    end
> +    local M = obj.month
> +    if M ~= nil then
> +        check_range(M, {1, 12}, 'month')
> +        ymd = true
> +    end
> +    local d = obj.day
> +    if d ~= nil then
> +        check_range(d, {1, 31}, 'day')

What happens if month = 2 and day = 31? This should raise an error,
I think.

> +        ymd = true
> +    end
> +    local h = obj.hour
> +    if h ~= nil then
> +        check_range(h, {0, 23}, 'hour')
> +        hms = true
> +    end
> +    local m = obj.min
> +    if m ~= nil then
> +        check_range(m, {0, 59}, 'min')
> +        hms = true
> +    end
> +    local ts = obj.sec

ts is typically short for 'timestamp', but here you deal with seconds in
a minute.

> +    local s = 0
> +    if ts ~= nil then
> +        check_range(ts, {0, 60}, 'sec')

Should be 59? There must be tests for every corner case.

> +        s, nsec = math_modf(ts)
> +        nsec = nsec * 1e9 -- convert fraction to nanoseconds
> +        hms = true
> +    end
> +    local offset = obj.tz
> +    if offset ~= nil then
> +        if type(offset) == 'number' then
> +            -- tz offset in minutes
> +            check_range(offset, {0, 720}, offset)
> +        elseif type(offset) == 'string' then
> +            local zone = parse_zone(offset)
> +            if zone == nil then
> +                error(('invalid time-zone format %s'):format(offset), 2)
> +            else
> +                offset = zone.offset
> +            end
> +        end
> +    end
> +
> +    -- .year, .month, .day
> +    if ymd then
> +        dt = builtin.tnt_dt_from_ymd(y or 0, M or 0, d or 0)
> +    end
> +
> +    -- .hour, .minute, .second
> +    local secs = 0
> +    if hms then
> +        secs = (h or 0) * 3600 + (m or 0) * 60 + (s or 0)
> +    end
> +
> +    return datetime_new_dt(dt, secs, nsec, offset)
> +end
> +
> +--[[
> +    Convert to text datetime values
> +
> +    - datetime will use ISO-8601 forat:
> +        1970-01-01T00:00Z
> +        2021-08-18T16:57:08.981725+03:00
> +]]

It looks like public functions should use different comment style:

https://www.tarantool.io/en/doc/latest/dev_guide/lua_style_guide/#commenting

> +local function datetime_tostring(o)
> +    if ffi.typeof(o) == datetime_t then
> +        local sz = 48
> +        local buff = ffi.new('char[?]', sz)

I think we've agreed that the buffer used for formatting should be
global.

> +        local len = builtin.datetime_to_string(o, buff, sz)
> +        assert(len < sz)
> +        return ffi.string(buff)
> +    end
> +end
> +
> +--[[
> +    Parse partial ISO-8601 date string
> +
> +    Accepetd formats are:
> +
> +    Basic      Extended
> +    20121224   2012-12-24   Calendar date   (ISO 8601)
> +    2012359    2012-359     Ordinal date    (ISO 8601)
> +    2012W521   2012-W52-1   Week date       (ISO 8601)
> +    2012Q485   2012-Q4-85   Quarter date
> +
> +    Returns pair of constructed datetime object, and length of string
> +    which has been accepted by parser.
> +]]
> +
> +local function parse_date(str)
> +    check_str("datetime.parse_date()")
> +    local dt = ffi.new('dt_t[1]')
> +    local len = builtin.tnt_dt_parse_iso_date(str, #str, dt)
> +    return len > 0 and datetime_new_dt(dt[0]) or nil, tonumber(len)
> +end
> +
> +--[[
> +    Basic    Extended
> +    Z        N/A
> +    +hh      N/A
> +    -hh      N/A
> +    +hhmm    +hh:mm
> +    -hhmm    -hh:mm
> +
> +    Returns pair of constructed datetime object, and length of string
> +    which has been accepted by parser.
> +]]
> +parse_zone = function(str)

local function datetime_parse_zone(str)

All datetime functions should be prefixed with datetime_ for consistency
and to avoid name clashes in future.

> +    check_str("datetime.parse_zone()")
> +    local offset = ffi.new('int[1]')
> +    local len = builtin.tnt_dt_parse_iso_zone_lenient(str, #str, offset)
> +    return len > 0 and offset[0], tonumber(len)
> +end
> +
> +--[[
> +    aggregated parse functions
> +    assumes to deal with date T time time_zone
> +    at once
> +
> +    date [T] time [ ] time_zone
> +
> +    Returns constructed datetime object.
> +]]
> +local function parse(str)
> +    check_str("datetime.parse()")
> +    local dt = ffi.new('dt_t[1]')
> +    local len = #str
> +    local n = builtin.tnt_dt_parse_iso_date(str, len, dt)
> +    local dt_ = dt[0]
> +    if n == 0 or len == n then
> +        return datetime_new_dt(dt_)
> +    end
> +
> +    str = str:sub(tonumber(n) + 1)
> +
> +    local ch = str:sub(1, 1)
> +    if ch:match('[Tt ]') == nil then
> +        return datetime_new_dt(dt_)
> +    end
> +
> +    str = str:sub(2)
> +    len = #str
> +
> +    local sp = ffi.new('int[1]')
> +    local fp = ffi.new('int[1]')
> +    local n = builtin.tnt_dt_parse_iso_time(str, len, sp, fp)
> +    if n == 0 then
> +        return datetime_new_dt(dt_)
> +    end
> +    local sp_ = sp[0]
> +    local fp_ = fp[0]
> +    if len == n then
> +        return datetime_new_dt(dt_, sp_, fp_)
> +    end
> +
> +    str = str:sub(tonumber(n) + 1)
> +
> +    if str:sub(1, 1) == ' ' then
> +        str = str:sub(2)
> +    end
> +
> +    len = #str
> +
> +    local offset = ffi.new('int[1]')
> +    n = builtin.tnt_dt_parse_iso_zone_lenient(str, len, offset)
> +    if n == 0 then
> +        return datetime_new_dt(dt_, sp_, fp_)
> +    end
> +    return datetime_new_dt(dt_, sp_, fp_, offset[0])
> +end
> +
> +--[[
> +    Dispatch function to create datetime from string or table.
> +    Creates default timeobject (pointing to Epoch date) if
> +    called without arguments.
> +]]
> +local function datetime_from(o)
> +    if o == nil or type(o) == 'table' then
> +        return datetime_new(o)
> +    elseif type(o) == 'string' then
> +        return parse(o)
> +    end
> +end
> +
> +--[[
> +    Create datetime object representing current time using microseconds
> +    platform timer and local timezone information.
> +]]
> +local function local_now()

datetime_now (all datetime methods should be prefixed with datetime_)

> +    local d = datetime_new_raw(0, 0, 0)
> +    builtin.datetime_now(d)
> +    return d
> +end
> +
> +-- addition or subtraction from date/time of a given interval
> +-- described via table direction should be +1 or -1

Please write a better comment: what expected in the argument; in what
order addition/subtraction proceeds.

> +local function datetime_increment(self, o, direction)

Confusing name: it's not increment. It's add or sub.

> +    assert(direction == -1 or direction == 1)
> +    local title = direction > 0 and "datetime.add" or "datetime.sub"
> +    check_date(self, title)
> +    if type(o) ~= 'table' then
> +        error(('%s - object expected'):format(title), 2)

Object? Or table?

> +    end
> +
> +    local secs, nsec = self.secs, self.nsec
> +    local offset = self.offset
> +
> +    -- operations with intervals should be done using human dates
> +    -- not UTC dates, thus we normalize to UTC
> +    local dt = local_dt(secs)
> +
> +    local ym_updated = false
> +    local years, months, weeks = o.years, o.months, o.weeks

Local variables are not really needed here.

> +
> +    if years ~= nil then
> +        check_range(years, {0, 9999}, 'years')

I think we should use the same names here (singular that is) as in
datetime.new(), to avoid confusion.

We can't create a year > 9999, but we can create year 9999 and add 1.
We should either remove the limitation or enforce it everywhere.

> +        dt = builtin.tnt_dt_add_years(dt, direction * years, builtin.DT_LIMIT)
> +        ym_updated = true
> +    end
> +    if months ~= nil then
> +        check_range(months, {0, 12}, 'months')

There must not be any limitations here. You can add 16 months or 48
hours to a date, it's perfrectly fine.

> +        dt = builtin.tnt_dt_add_months(dt, direction * months, builtin.DT_LIMIT)

I'd really prefer to hide this dt thing behind datetime lib so that here
we would only operate on datetime object.

> +        ym_updated = true
> +    end
> +    if ym_updated then
> +        secs = (builtin.tnt_dt_rdn(dt) - DT_EPOCH_1970_OFFSET) * SECS_PER_DAY +
> +                secs % SECS_PER_DAY
> +    end
> +
> +    if weeks ~= nil then
> +        check_range(weeks, {0, 52}, 'weeks')
> +        secs = secs + direction * 7 * weeks * SECS_PER_DAY
> +    end
> +
> +    local days, hours, minutes, seconds = o.days, o.hours, o.minutes, o.seconds
> +    if days ~= nil then
> +        check_range(days, {0, 31}, 'days')
> +        secs = secs + direction * days * SECS_PER_DAY
> +    end
> +    if hours ~= nil then
> +        check_range(hours, {0, 23}, 'hours')
> +        secs = secs + direction * 60 * 60 * hours
> +    end
> +    if minutes ~= nil then
> +        check_range(minutes, {0, 59}, 'minutes')
> +        secs = secs + direction * 60 * minutes
> +    end
> +    if seconds ~= nil then
> +        check_range(seconds, {0, 60}, 'seconds')
> +        local s, frac = math.modf(seconds)
> +        secs = secs + direction * s
> +        nsec = nsec + direction * frac * 1e9
> +    end
> +
> +    secs, nsec = normalize_nsec(secs, nsec)
> +
> +    return datetime_new_raw(secs, nsec, offset)
> +end
> +
> +--[[
> +    Return table in os.date('*t') format, but with timezone
> +    and nanoseconds
> +]]
> +local function datetime_totable(self)
> +    local secs = self.secs
> +    local dt = local_dt(secs)
> +    local year = builtin.dt_year(dt)
> +    local month = builtin.dt_month(dt)
> +    local yday = builtin.dt_doy(dt)
> +    local wday = ffi.cast('int32_t', builtin.tnt_dt_dow(dt))
> +    local day_of_month = builtin.dt_dom(dt)
> +    local hour = math_floor((secs / 3600) % 24)
> +    local minute = math_floor((secs / 60) % 60)
> +    local second = secs % 60

All these local variables only clutter the code - please initialize the
table directly.

> +
> +    return {
> +        sec = second,
> +        min = minute,
> +        day = day_of_month,
> +        isdst = false,
> +        wday = wday,
> +        yday = yday,
> +        year = year,
> +        month = month,
> +        hour = hour,
> +        nsec = self.nsec,

You return 'nsec' here, but neither datetime_new nor add/sub take 'nsec'
AFAICS. Should we add 'nsec' to new/add/sub and ignore fractional part
of 'sec' in those methods?

> +        tz = self.offset,
> +    }
> +end
> +
> +local function datetime_set(self, obj)
> +    -- FIXME

Please fix.

> +    return datetime_new(obj)
> +end
> +
> +local function strftime(fmt, o)
> +    check_date(o, "datetime.strftime()")
> +    local sz = 128
> +    local buff = ffi.new('char[?]', sz)
> +    builtin.datetime_strftime(o, fmt, buff, sz)
> +    return ffi.string(buff)
> +end
> +
> +ffi.metatype(datetime_t, {
> +    __tostring = datetime_tostring,
> +    __serialize = datetime_serialize,
> +    __eq = datetime_eq,
> +    __lt = datetime_lt,
> +    __le = datetime_le,
> +    __index = {
> +        epoch = function(self) return self.secs end,
> +        timestamp = function(self) return self.secs + self.nsec / 1e9 end,

Please let's not mix lambdas and functions here - let's define function
wrappers for consistency.

> +        nanoseconds = function(self) return self.secs * 1e9 + self.nsec end,
> +        microseconds = function(self) return self.secs * 1e6 + self.nsec / 1e3 end,
> +        milliseconds = function(self) return self.secs * 1e3 + self.nsec / 1e6 end,
> +        seconds = function(self) return self.secs + self.nsec / 1e9 end,

Please remove these four methods. There should only be epoch() and
timestamp() - the user can convert them to nanoseconds, microseconds,
whatever by themselves.

> +        add = function(self, obj) return datetime_increment(self, obj, 1) end,
> +        sub = function(self, obj) return datetime_increment(self, obj, -1) end,

Please implement the 'set' method requested by @unera.

> +        totable = datetime_totable,
> +        set = datetime_set,
> +    }

Please don't use __index. Just put the methods in the metatable.

> +})
> +
> +return setmetatable(
> +    {
> +        new         = datetime_new,
> +        new_raw     = datetime_new_obj,

new_raw is a bad name. I think datetime.new() should handle this:

datetime.new{epoch = ..., tz = ... }

or

datetime.new{timestamp = ..., tz = ... }

> +
> +        parse       = parse,
> +        parse_date  = parse_date,

I think we've agreed to remove this fuctions from the public API.
Parsing should be handled by datetime.new().

> +
> +        now         = local_now,
> +        strftime    = strftime,

strftime should be a method of datetime object.

> +
> +        is_datetime = is_datetime,
> +    }, {
> +        __call = function(self, ...) return datetime_from(...) end
> +    }

I'd expect datetime() to be a synonym for datetime.new(), but hey seem
to be different. Anyway, I don't think we really need to support
datetime() - datetime.new() should be enough.

> +)
> diff --git a/src/lua/init.c b/src/lua/init.c
> index f9738025d..127e935d7 100644
> --- a/src/lua/init.c
> +++ b/src/lua/init.c
> @@ -129,7 +129,8 @@ extern char strict_lua[],
>  	parse_lua[],
>  	process_lua[],
>  	humanize_lua[],
> -	memprof_lua[]
> +	memprof_lua[],
> +	datetime_lua[]
>  ;
>  
>  static const char *lua_modules[] = {
> @@ -184,6 +185,7 @@ static const char *lua_modules[] = {
>  	"memprof.process", process_lua,
>  	"memprof.humanize", humanize_lua,
>  	"memprof", memprof_lua,
> +	"datetime", datetime_lua,
>  	NULL
>  };
>  
> diff --git a/src/lua/utils.c b/src/lua/utils.c
> index c71cd4857..2c89326f3 100644
> --- a/src/lua/utils.c
> +++ b/src/lua/utils.c
> @@ -48,6 +48,9 @@ static uint32_t CTID_STRUCT_IBUF_PTR;
>  uint32_t CTID_CHAR_PTR;
>  uint32_t CTID_CONST_CHAR_PTR;
>  uint32_t CTID_UUID;
> +uint32_t CTID_DATETIME = 0;

This belongs to the next patch - it isn't used in this patch.

> +uint32_t CTID_INTERVAL = 0;

No such thing anymore.

> +

Extra new line.

>  
>  void *
>  luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
> @@ -120,6 +123,12 @@ luaL_pushuuidstr(struct lua_State *L, const struct tt_uuid *uuid)
>  	lua_pushlstring(L, str, UUID_STR_LEN);
>  }
>  
> +struct datetime *
> +luaL_pushdatetime(struct lua_State *L)
> +{
> +	return luaL_pushcdata(L, CTID_DATETIME);
> +}
> +
>  int
>  luaL_iscdata(struct lua_State *L, int idx)
>  {
> @@ -725,6 +734,24 @@ tarantool_lua_utils_init(struct lua_State *L)
>  	CTID_UUID = luaL_ctypeid(L, "struct tt_uuid");
>  	assert(CTID_UUID != 0);
>  
> +	rc = luaL_cdef(L, "struct datetime {"
> +			  "double secs;"
> +			  "int32_t nsec;"

It is uint32_t in struct datetime now.

> +			  "int32_t offset;"
> +			  "};");
> +	assert(rc == 0);
> +	(void) rc;
> +	CTID_DATETIME = luaL_ctypeid(L, "struct datetime");
> +	assert(CTID_DATETIME != 0);
> +	rc = luaL_cdef(L, "struct datetime_interval {"
> +			  "double secs;"
> +			  "int32_t nsec;"
> +			  "};");
> +	assert(rc == 0);
> +	(void) rc;
> +	CTID_INTERVAL = luaL_ctypeid(L, "struct datetime_interval");
> +	assert(CTID_INTERVAL != 0);
> +
>  	lua_pushcfunction(L, luaT_newthread_wrapper);
>  	luaT_newthread_ref = luaL_ref(L, LUA_REGISTRYINDEX);
>  	return 0;
> diff --git a/src/lua/utils.h b/src/lua/utils.h
> index 45070b778..73495b607 100644
> --- a/src/lua/utils.h
> +++ b/src/lua/utils.h
> @@ -59,6 +59,7 @@ struct lua_State;
>  struct ibuf;
>  typedef struct ibuf box_ibuf_t;
>  struct tt_uuid;
> +struct datetime;
>  
>  /**
>   * Single global lua_State shared by core and modules.
> @@ -71,6 +72,8 @@ extern struct lua_State *tarantool_L;
>  extern uint32_t CTID_CHAR_PTR;
>  extern uint32_t CTID_CONST_CHAR_PTR;
>  extern uint32_t CTID_UUID;
> +extern uint32_t CTID_DATETIME;
> +extern uint32_t CTID_INTERVAL;
>  
>  struct tt_uuid *
>  luaL_pushuuid(struct lua_State *L);
> @@ -78,6 +81,15 @@ luaL_pushuuid(struct lua_State *L);
>  void
>  luaL_pushuuidstr(struct lua_State *L, const struct tt_uuid *uuid);
>  
> +/**
> + * @brief Push cdata of a datetime type onto the stack.
> + * @param L Lua State
> + * @sa luaL_pushcdata
> + * @return memory associated with this datetime data
> + */
> +struct datetime *
> +luaL_pushdatetime(struct lua_State *L);
> +
>  /** \cond public */
>  
>  /**
> diff --git a/test/app-tap/datetime.test.lua b/test/app-tap/datetime.test.lua
> new file mode 100755
> index 000000000..5c39feee8
> --- /dev/null
> +++ b/test/app-tap/datetime.test.lua
> @@ -0,0 +1,213 @@
> +#!/usr/bin/env tarantool
> +
> +local tap = require('tap')
> +local test = tap.test("errno")
> +local date = require('datetime')
> +local ffi = require('ffi')
> +
> +test:plan(7)
> +
> +test:test("Simple tests for parser", function(test)
> +    test:plan(2)
> +    test:ok(date("1970-01-01T01:00:00Z") ==
> +            date {year=1970, month=1, day=1, hour=1, minute=0, second=0})
> +    test:ok(date("1970-01-01T02:00:00+02:00") ==
> +            date {year=1970, month=1, day=1, hour=2, minute=0, second=0, tz=120})
> +end)
> +
> +test:test("Multiple tests for parser (with nanoseconds)", function(test)
> +    test:plan(193)
> +    -- borrowed from p5-time-moments/t/180_from_string.t
> +    local tests =
> +    {
> +        {'1970-01-01T00:00Z',                  0,         0,    0, 1},
> +        {'1970-01-01T02:00+02:00',             0,         0,  120, 1},
> +        {'1970-01-01T01:30+01:30',             0,         0,   90, 1},
> +        {'1970-01-01T01:00+01:00',             0,         0,   60, 1},
> +        {'1970-01-01T00:01+00:01',             0,         0,    1, 1},
> +        {'1970-01-01T00:00Z',                  0,         0,    0, 1},
> +        {'1969-12-31T23:59-00:01',             0,         0,   -1, 1},
> +        {'1969-12-31T23:00-01:00',             0,         0,  -60, 1},
> +        {'1969-12-31T22:30-01:30',             0,         0,  -90, 1},
> +        {'1969-12-31T22:00-02:00',             0,         0, -120, 1},
> +        {'1970-01-01T00:00:00.123456789Z',     0, 123456789,    0, 1},
> +        {'1970-01-01T00:00:00.12345678Z',      0, 123456780,    0, 0},
> +        {'1970-01-01T00:00:00.1234567Z',       0, 123456700,    0, 0},
> +        {'1970-01-01T00:00:00.123456Z',        0, 123456000,    0, 1},
> +        {'1970-01-01T00:00:00.12345Z',         0, 123450000,    0, 0},
> +        {'1970-01-01T00:00:00.1234Z',          0, 123400000,    0, 0},
> +        {'1970-01-01T00:00:00.123Z',           0, 123000000,    0, 1},
> +        {'1970-01-01T00:00:00.12Z',            0, 120000000,    0, 0},
> +        {'1970-01-01T00:00:00.1Z',             0, 100000000,    0, 0},
> +        {'1970-01-01T00:00:00.01Z',            0,  10000000,    0, 0},
> +        {'1970-01-01T00:00:00.001Z',           0,   1000000,    0, 1},
> +        {'1970-01-01T00:00:00.0001Z',          0,    100000,    0, 0},
> +        {'1970-01-01T00:00:00.00001Z',         0,     10000,    0, 0},
> +        {'1970-01-01T00:00:00.000001Z',        0,      1000,    0, 1},
> +        {'1970-01-01T00:00:00.0000001Z',       0,       100,    0, 0},
> +        {'1970-01-01T00:00:00.00000001Z',      0,        10,    0, 0},
> +        {'1970-01-01T00:00:00.000000001Z',     0,         1,    0, 1},
> +        {'1970-01-01T00:00:00.000000009Z',     0,         9,    0, 1},
> +        {'1970-01-01T00:00:00.00000009Z',      0,        90,    0, 0},
> +        {'1970-01-01T00:00:00.0000009Z',       0,       900,    0, 0},
> +        {'1970-01-01T00:00:00.000009Z',        0,      9000,    0, 1},
> +        {'1970-01-01T00:00:00.00009Z',         0,     90000,    0, 0},
> +        {'1970-01-01T00:00:00.0009Z',          0,    900000,    0, 0},
> +        {'1970-01-01T00:00:00.009Z',           0,   9000000,    0, 1},
> +        {'1970-01-01T00:00:00.09Z',            0,  90000000,    0, 0},
> +        {'1970-01-01T00:00:00.9Z',             0, 900000000,    0, 0},
> +        {'1970-01-01T00:00:00.99Z',            0, 990000000,    0, 0},
> +        {'1970-01-01T00:00:00.999Z',           0, 999000000,    0, 1},
> +        {'1970-01-01T00:00:00.9999Z',          0, 999900000,    0, 0},
> +        {'1970-01-01T00:00:00.99999Z',         0, 999990000,    0, 0},
> +        {'1970-01-01T00:00:00.999999Z',        0, 999999000,    0, 1},
> +        {'1970-01-01T00:00:00.9999999Z',       0, 999999900,    0, 0},
> +        {'1970-01-01T00:00:00.99999999Z',      0, 999999990,    0, 0},
> +        {'1970-01-01T00:00:00.999999999Z',     0, 999999999,    0, 1},
> +        {'1970-01-01T00:00:00.0Z',             0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.00Z',            0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.000Z',           0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.0000Z',          0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.00000Z',         0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.000000Z',        0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.0000000Z',       0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.00000000Z',      0,         0,    0, 0},
> +        {'1970-01-01T00:00:00.000000000Z',     0,         0,    0, 0},
> +        {'1973-11-29T21:33:09Z',       123456789,         0,    0, 1},
> +        {'2013-10-28T17:51:56Z',      1382982716,         0,    0, 1},
> +        {'9999-12-31T23:59:59Z',    253402300799,         0,    0, 1},
> +    }
> +    for _, value in ipairs(tests) do
> +        local str, epoch, nsec, offset, check
> +        str, epoch, nsec, offset, check = unpack(value)
> +        local dt = date(str)
> +        test:ok(dt.secs == epoch, ('%s: dt.secs == %d'):format(str, epoch))

Shouldn't be that easily accessible from the code. Prepend with an
underscore to emphasize that they are private members?

> +        test:ok(dt.nsec == nsec, ('%s: dt.nsec == %d'):format(str, nsec))
> +        test:ok(dt.offset == offset, ('%s: dt.offset == %d'):format(str, offset))
> +        if check > 0 then
> +            test:ok(str == tostring(dt), ('%s == tostring(%s)'):
> +                    format(str, tostring(dt)))
> +        end
> +    end
> +end)
> +
> +ffi.cdef [[
> +    void tzset(void);
> +]]

Unused?

> +
> +test:test("Datetime string formatting", function(test)
> +    test:plan(5)
> +    local str = "1970-01-01"
> +    local t = date(str)
> +    test:ok(t.secs == 0, ('%s: t.secs == %d'):format(str, tonumber(t.secs)))
> +    test:ok(t.nsec == 0, ('%s: t.nsec == %d'):format(str, t.nsec))
> +    test:ok(t.offset == 0, ('%s: t.offset == %d'):format(str, t.offset))
> +    test:ok(date.strftime('%d/%m/%Y', t) == '01/01/1970', ('%s: strftime #1'):format(str))
> +    test:ok(date.strftime('%A %d. %B %Y', t) == 'Thursday 01. January 1970', ('%s: strftime #2'):format(str))
> +end)
> +
> +test:test("Parse iso date - valid strings", function(test)
> +    test:plan(32)
> +    local good = {
> +        {2012, 12, 24, "20121224",                   8 },
> +        {2012, 12, 24, "20121224  Foo bar",          8 },
> +        {2012, 12, 24, "2012-12-24",                10 },
> +        {2012, 12, 24, "2012-12-24 23:59:59",       10 },
> +        {2012, 12, 24, "2012-12-24T00:00:00+00:00", 10 },
> +        {2012, 12, 24, "2012359",                    7 },
> +        {2012, 12, 24, "2012359T235959+0130",        7 },
> +        {2012, 12, 24, "2012-359",                   8 },
> +        {2012, 12, 24, "2012W521",                   8 },
> +        {2012, 12, 24, "2012-W52-1",                10 },
> +        {2012, 12, 24, "2012Q485",                   8 },
> +        {2012, 12, 24, "2012-Q4-85",                10 },
> +        {   1,  1,  1, "0001-Q1-01",                10 },
> +        {   1,  1,  1, "0001-W01-1",                10 },
> +        {   1,  1,  1, "0001-01-01",                10 },
> +        {   1,  1,  1, "0001-001",                   8 },
> +    }
> +
> +    for _, value in ipairs(good) do
> +        local year, month, day, str, date_part_len
> +        year, month, day, str, date_part_len = unpack(value)
> +        local expected_date = date{year = year, month = month, day = day}
> +        local date_part, len
> +        date_part, len = date.parse_date(str)
> +        test:ok(len == date_part_len, ('%s: length check %d'):format(str, len))
> +        test:ok(expected_date == date_part, ('%s: expected date'):format(str))
> +    end
> +end)
> +
> +test:test("Parse iso date - invalid strings", function(test)
> +    test:plan(62)
> +    local bad = {
> +        "20121232"   , -- Invalid day of month
> +        "2012-12-310", -- Invalid day of month
> +        "2012-13-24" , -- Invalid month
> +        "2012367"    , -- Invalid day of year
> +        "2012-000"   , -- Invalid day of year
> +        "2012W533"   , -- Invalid week of year
> +        "2012-W52-8" , -- Invalid day of week
> +        "2012Q495"   , -- Invalid day of quarter
> +        "2012-Q5-85" , -- Invalid quarter
> +        "20123670"   , -- Trailing digit
> +        "201212320"  , -- Trailing digit
> +        "2012-12"    , -- Reduced accuracy
> +        "2012-Q4"    , -- Reduced accuracy
> +        "2012-Q42"   , -- Invalid
> +        "2012-Q1-1"  , -- Invalid day of quarter
> +        "2012Q--420" , -- Invalid
> +        "2012-Q-420" , -- Invalid
> +        "2012Q11"    , -- Incomplete
> +        "2012Q1234"  , -- Trailing digit
> +        "2012W12"    , -- Incomplete
> +        "2012W1234"  , -- Trailing digit
> +        "2012W-123"  , -- Invalid
> +        "2012-W12"   , -- Incomplete
> +        "2012-W12-12", -- Trailing digit
> +        "2012U1234"  , -- Invalid
> +        "2012-1234"  , -- Invalid
> +        "2012-X1234" , -- Invalid
> +        "0000-Q1-01" , -- Year less than 0001
> +        "0000-W01-1" , -- Year less than 0001
> +        "0000-01-01" , -- Year less than 0001
> +        "0000-001"   , -- Year less than 0001
> +    }
> +
> +    for _, str in ipairs(bad) do
> +        local date_part, len
> +        date_part, len = date.parse_date(str)
> +        test:ok(len == 0, ('%s: length check %d'):format(str, len))
> +        test:ok(date_part == nil, ('%s: empty date check %s'):format(str, date_part))
> +    end
> +end)
> +
> +test:test("Parse tiny date into seconds and other parts", function(test)
> +    test:plan(7)
> +    local str = '19700101 00:00:30.528'
> +    local tiny = date(str)
> +    test:ok(tiny.secs == 30, ("secs of '%s'"):format(str))
> +    test:ok(tiny.nsec == 528000000, ("nsec of '%s'"):format(str))
> +    test:ok(tiny:nanoseconds() == 30528000000, "nanoseconds")
> +    test:ok(tiny:microseconds() == 30528000, "microseconds")
> +    test:ok(tiny:milliseconds() == 30528, "milliseconds")
> +    test:ok(tiny:seconds() == 30.528, "seconds")
> +    test:ok(tiny:timestamp() == 30.528, "timestamp")
> +end)
> +
> +test:test("Time interval operations", function(test)
> +    test:plan(2)
> +
> +    -- check arithmetic with leap dates
> +    local T = date('1972-02-29')
> +    test:ok(tostring(T:add{years = 1, months = 2}) == '1973-05-01T00:00Z',
> +            ('T:add{years=1,months=2}(%s)'):format(T))
> +
> +    -- check average, not leap dates
> +    T = date('1970-01-08')
> +    test:ok(tostring(T:add{years = 1, months = 2}) == '1971-03-08T00:00Z',
> +            ('T:add{years=1,months=2}(%s)'):format(T))
> +
> +end)
> +
> +os.exit(test:check() and 0 or 1)
> diff --git a/test/unit/CMakeLists.txt b/test/unit/CMakeLists.txt
> index 5bb7cd6e7..8194a133f 100644
> --- a/test/unit/CMakeLists.txt
> +++ b/test/unit/CMakeLists.txt
> @@ -56,7 +56,8 @@ add_executable(uuid.test uuid.c core_test_utils.c)
>  target_link_libraries(uuid.test uuid unit)
>  add_executable(random.test random.c core_test_utils.c)
>  target_link_libraries(random.test core unit)
> -
> +add_executable(datetime.test datetime.c)
> +target_link_libraries(datetime.test cdt core unit)
>  add_executable(bps_tree.test bps_tree.cc)
>  target_link_libraries(bps_tree.test small misc)
>  add_executable(bps_tree_iterator.test bps_tree_iterator.cc)
> diff --git a/test/unit/datetime.c b/test/unit/datetime.c
> new file mode 100644
> index 000000000..931636172
> --- /dev/null
> +++ b/test/unit/datetime.c
> @@ -0,0 +1,260 @@
> +#include "dt.h"
> +#include <assert.h>
> +#include <stdint.h>
> +#include <string.h>
> +#include <time.h>
> +
> +#include "unit.h"
> +#include "datetime.h"
> +#include "trivia/util.h"
> +
> +static const char sample[] = "2012-12-24T15:30Z";
> +
> +#define S(s) {s, sizeof(s) - 1}
> +struct {
> +	const char *str;
> +	size_t len;
> +} tests[] = {
> +	S("2012-12-24 15:30Z"),
> +	S("2012-12-24 15:30z"),
> +	S("2012-12-24 15:30"),
> +	S("2012-12-24 16:30+01:00"),
> +	S("2012-12-24 16:30+0100"),
> +	S("2012-12-24 16:30+01"),
> +	S("2012-12-24 14:30-01:00"),
> +	S("2012-12-24 14:30-0100"),
> +	S("2012-12-24 14:30-01"),
> +	S("2012-12-24 15:30:00Z"),
> +	S("2012-12-24 15:30:00z"),
> +	S("2012-12-24 15:30:00"),
> +	S("2012-12-24 16:30:00+01:00"),
> +	S("2012-12-24 16:30:00+0100"),
> +	S("2012-12-24 14:30:00-01:00"),
> +	S("2012-12-24 14:30:00-0100"),
> +	S("2012-12-24 15:30:00.123456Z"),
> +	S("2012-12-24 15:30:00.123456z"),
> +	S("2012-12-24 15:30:00.123456"),
> +	S("2012-12-24 16:30:00.123456+01:00"),
> +	S("2012-12-24 16:30:00.123456+01"),
> +	S("2012-12-24 14:30:00.123456-01:00"),
> +	S("2012-12-24 14:30:00.123456-01"),
> +	S("2012-12-24t15:30Z"),
> +	S("2012-12-24t15:30z"),
> +	S("2012-12-24t15:30"),
> +	S("2012-12-24t16:30+01:00"),
> +	S("2012-12-24t16:30+0100"),
> +	S("2012-12-24t14:30-01:00"),
> +	S("2012-12-24t14:30-0100"),
> +	S("2012-12-24t15:30:00Z"),
> +	S("2012-12-24t15:30:00z"),
> +	S("2012-12-24t15:30:00"),
> +	S("2012-12-24t16:30:00+01:00"),
> +	S("2012-12-24t16:30:00+0100"),
> +	S("2012-12-24t14:30:00-01:00"),
> +	S("2012-12-24t14:30:00-0100"),
> +	S("2012-12-24t15:30:00.123456Z"),
> +	S("2012-12-24t15:30:00.123456z"),
> +	S("2012-12-24t16:30:00.123456+01:00"),
> +	S("2012-12-24t14:30:00.123456-01:00"),
> +	S("2012-12-24 16:30 +01:00"),
> +	S("2012-12-24 14:30 -01:00"),
> +	S("2012-12-24 15:30 UTC"),
> +	S("2012-12-24 16:30 UTC+1"),
> +	S("2012-12-24 16:30 UTC+01"),
> +	S("2012-12-24 16:30 UTC+0100"),
> +	S("2012-12-24 16:30 UTC+01:00"),
> +	S("2012-12-24 14:30 UTC-1"),
> +	S("2012-12-24 14:30 UTC-01"),
> +	S("2012-12-24 14:30 UTC-01:00"),
> +	S("2012-12-24 14:30 UTC-0100"),
> +	S("2012-12-24 15:30 GMT"),
> +	S("2012-12-24 16:30 GMT+1"),
> +	S("2012-12-24 16:30 GMT+01"),
> +	S("2012-12-24 16:30 GMT+0100"),
> +	S("2012-12-24 16:30 GMT+01:00"),
> +	S("2012-12-24 14:30 GMT-1"),
> +	S("2012-12-24 14:30 GMT-01"),
> +	S("2012-12-24 14:30 GMT-01:00"),
> +	S("2012-12-24 14:30 GMT-0100"),
> +	S("2012-12-24 14:30 -01:00"),
> +	S("2012-12-24 16:30:00 +01:00"),
> +	S("2012-12-24 14:30:00 -01:00"),
> +	S("2012-12-24 16:30:00.123456 +01:00"),
> +	S("2012-12-24 14:30:00.123456 -01:00"),
> +	S("2012-12-24 15:30:00.123456 -00:00"),
> +	S("20121224T1630+01:00"),
> +	S("2012-12-24T1630+01:00"),
> +	S("20121224T16:30+01"),
> +	S("20121224T16:30 +01"),
> +};
> +#undef S
> +
> +static int
> +parse_datetime(const char *str, size_t len, int64_t *secs_p,
> +	       int32_t *nanosecs_p, int32_t *offset_p)
> +{
> +	size_t n;
> +	dt_t dt;
> +	char c;
> +	int sec_of_day = 0, nanosecond = 0, offset = 0;
> +
> +	n = dt_parse_iso_date(str, len, &dt);
> +	if (!n)
> +		return 1;
> +	if (n == len)
> +		goto exit;
> +
> +	c = str[n++];
> +	if (!(c == 'T' || c == 't' || c == ' '))
> +		return 1;
> +
> +	str += n;
> +	len -= n;
> +
> +	n = dt_parse_iso_time(str, len, &sec_of_day, &nanosecond);
> +	if (!n)
> +		return 1;
> +	if (n == len)
> +		goto exit;
> +
> +	if (str[n] == ' ')
> +		n++;
> +
> +	str += n;
> +	len -= n;
> +
> +	n = dt_parse_iso_zone_lenient(str, len, &offset);
> +	if (!n || n != len)
> +		return 1;
> +
> +exit:
> +	*secs_p = ((int64_t)dt_rdn(dt) - DT_EPOCH_1970_OFFSET) * SECS_PER_DAY +
> +		  sec_of_day - offset * 60;
> +	*nanosecs_p = nanosecond;
> +	*offset_p = offset;
> +
> +	return 0;
> +}
> +
> +static int
> +local_rd(const struct datetime *dt)
> +{
> +	return (int)((int64_t)dt->secs / SECS_PER_DAY) + DT_EPOCH_1970_OFFSET;
> +}
> +
> +static int
> +local_dt(const struct datetime *dt)
> +{
> +	return dt_from_rdn(local_rd(dt));
> +}
> +
> +struct tm *
> +datetime_to_tm(struct datetime *dt)
> +{
> +	static struct tm tm;
> +
> +	memset(&tm, 0, sizeof(tm));
> +	dt_to_struct_tm(local_dt(dt), &tm);
> +
> +	int seconds_of_day = (int64_t)dt->secs % 86400;
> +	tm.tm_hour = (seconds_of_day / 3600) % 24;
> +	tm.tm_min = (seconds_of_day / 60) % 60;
> +	tm.tm_sec = seconds_of_day % 60;
> +
> +	return &tm;
> +}

Code duplication with the code from the datetime module. I think that
all datetime helper functions should be defined only in the datetime
module and used both by the code and the tests.

> +
> +static void datetime_test(void)
> +{
> +	size_t index;
> +	int64_t secs_expected;
> +	int32_t nanosecs;
> +	int32_t offset;
> +
> +	plan(355);
> +	parse_datetime(sample, sizeof(sample) - 1,
> +		       &secs_expected, &nanosecs, &offset);
> +
> +	for (index = 0; index < lengthof(tests); index++) {
> +		int64_t secs;
> +		int rc = parse_datetime(tests[index].str, tests[index].len,
> +					&secs, &nanosecs, &offset);
> +		is(rc, 0, "correct parse_datetime return value for '%s'",
> +		   tests[index].str);
> +		is(secs, secs_expected, "correct parse_datetime output "
> +					"seconds for '%s",
> +		   tests[index].str);
> +
> +		/*
> +		 * check that stringized literal produces the same date
> +		 * time fields
> +		 */
> +		static char buff[40];
> +		struct datetime dt = {secs, nanosecs, offset};
> +		/* datetime_to_tm returns time in GMT zone */
> +		struct tm *p_tm = datetime_to_tm(&dt);
> +		size_t len = strftime(buff, sizeof(buff), "%F %T", p_tm);
> +		ok(len > 0, "strftime");
> +		int64_t parsed_secs;
> +		int32_t parsed_nsecs, parsed_ofs;
> +		rc = parse_datetime(buff, len, &parsed_secs, &parsed_nsecs, &parsed_ofs);
> +		is(rc, 0, "correct parse_datetime return value for '%s'", buff);
> +		is(secs, parsed_secs,
> +		   "reversible seconds via strftime for '%s", buff);
> +	}
> +	check_plan();
> +}
> +
> +
> +static void
> +tostring_datetime_test(void)
> +{
> +	static struct {
> +		const char *string;
> +		int64_t     secs;
> +		uint32_t    nsec;
> +		uint32_t    offset;
> +	} tests[] = {
> +		{"1970-01-01T02:00+02:00",          0,         0,  120},
> +		{"1970-01-01T01:30+01:30",          0,         0,   90},
> +		{"1970-01-01T01:00+01:00",          0,         0,   60},
> +		{"1970-01-01T00:01+00:01",          0,         0,    1},
> +		{"1970-01-01T00:00Z",               0,         0,    0},
> +		{"1969-12-31T23:59-00:01",          0,         0,   -1},
> +		{"1969-12-31T23:00-01:00",          0,         0,  -60},
> +		{"1969-12-31T22:30-01:30",          0,         0,  -90},
> +		{"1969-12-31T22:00-02:00",          0,         0, -120},
> +		{"1970-01-01T00:00:00.123456789Z",  0, 123456789,    0},
> +		{"1970-01-01T00:00:00.123456Z",     0, 123456000,    0},
> +		{"1970-01-01T00:00:00.123Z",        0, 123000000,    0},
> +		{"1973-11-29T21:33:09Z",    123456789,         0,    0},
> +		{"2013-10-28T17:51:56Z",   1382982716,         0,    0},
> +		{"9999-12-31T23:59:59Z", 253402300799,         0,    0},
> +	};
> +	size_t index;
> +
> +	plan(15);
> +	for (index = 0; index < lengthof(tests); index++) {
> +		struct datetime date = {
> +			tests[index].secs,
> +			tests[index].nsec,
> +			tests[index].offset
> +		};
> +		char buf[48];
> +		datetime_to_string(&date, buf, sizeof(buf));
> +		is(strcmp(buf, tests[index].string), 0,
> +		   "string '%s' expected, received '%s'",
> +		   tests[index].string, buf);
> +	}
> +	check_plan();
> +}
> +
> +int
> +main(void)
> +{
> +	plan(2);
> +	datetime_test();
> +	tostring_datetime_test();
> +
> +	return check_plan();
> +}

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser
  2021-08-19 10:29     ` Safin Timur via Tarantool-patches
  2021-08-19 11:11       ` Serge Petrenko via Tarantool-patches
@ 2021-08-19 15:58       ` Vladimir Davydov via Tarantool-patches
  1 sibling, 0 replies; 20+ messages in thread
From: Vladimir Davydov via Tarantool-patches @ 2021-08-19 15:58 UTC (permalink / raw)
  To: Safin Timur; +Cc: tarantool-patches, v.shpilevoy

On Thu, Aug 19, 2021 at 01:29:06PM +0300, Safin Timur wrote:
> On 19.08.2021 13:19, Serge Petrenko wrote:
> > 
> > 
> > 19.08.2021 05:56, Timur Safin пишет:
> > > It was told that if field `datetime.secs` would be `double` we should get
> > > better performance in LuaJIT instead of `uint64_t` type, which is
> > > used at the
> > > moment.
> > > 
> > > So we have created benchmark, which was comparing implementations of
> > > functions
> > > from `datetime.c` if we would use `double` or `int64_t` for
> > > `datetime.secs` field.
> > > 
> > > Despite expectations, based on prior experience with floaing-point on x86
> > > processors, comparison shows that `double` provides similar or
> > > sometimes better timings. And picture stays consistent be it SSE2,
> > > AVX1 or
> > > AVX2 code.
> > > 
> > > Part of #5941
> > > ---
> > 
> > I agree with Vladimir here.
> > Looks like this perf test doesn't belong to Tarantool repository.
> > Would you mind dropping it?
> 
> Here is the case (we both aware of) I want to avoid here - today we do not
> have saved _that_ decimal perf test, basing on which we have preferred LuaC
> and dropped FFI implementation. We could not rerun it today, within a newer
> LuaJIT implementation, to verify that situation didn't change. This is
> similar case - we have made a decision basing on some evaluations using this
> code, in a future we may decide to further optimize data structure (like
> Vova suggested elsewhere to split int64 into 2 fields), and it would be
> better if at that moment we would still have performance test around for
> adaptations and rerun.
> 
> Yes, it's another test of performance test we used to see in perf directory
> (hehe, there is only single test at the moment), kind of one time shot in a
> history, important for design decision, but from longer prospective I assume
> it should be still around.
> 
> Does my reasoning make some sense?

A test that forks parts of Tarantool internal code and runs some
benchmarks on them (like this one) will inevitably diverge from the code
it was forked from. So pretty soon its results won't be trusted. Having
it in the main repository is a maintenance burden.

A test that checks FFI vs ccall performance or double vs int64_t should
be independent on Tarantool internals. Such a test is probably okay to
have in the main repository, although it could just as well live in a
separate repository with performance tests, but this is a matter of
policy.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime
  2021-08-19 15:26   ` Vladimir Davydov via Tarantool-patches
@ 2021-08-24 21:13     ` Vladislav Shpilevoy via Tarantool-patches
  0 siblings, 0 replies; 20+ messages in thread
From: Vladislav Shpilevoy via Tarantool-patches @ 2021-08-24 21:13 UTC (permalink / raw)
  To: Vladimir Davydov, Timur Safin; +Cc: tarantool-patches

>> +local function datetime_tostring(o)
>> +    if ffi.typeof(o) == datetime_t then
>> +        local sz = 48
>> +        local buff = ffi.new('char[?]', sz)
> 
> I think we've agreed that the buffer used for formatting should be
> global.

If you are going with a global buffer, please, take into
account Lua GC and tickets 5632, 6050, 6259. If there is
going to be a global buffer, it must be properly shared. For
example, using buffer.ffi_stash_new(). See it source and
usages to find what that is.

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2021-08-24 21:13 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-19  2:56 [Tarantool-patches] [PATCH v6 0/5] Initial datetime implementation Timur Safin via Tarantool-patches
2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 1/5] build, lua: built-in module datetime Timur Safin via Tarantool-patches
2021-08-19  9:43   ` Serge Petrenko via Tarantool-patches
2021-08-19  9:47     ` Safin Timur via Tarantool-patches
2021-08-19 15:26   ` Vladimir Davydov via Tarantool-patches
2021-08-24 21:13     ` Vladislav Shpilevoy via Tarantool-patches
2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 2/5] box, datetime: messagepack support for datetime Timur Safin via Tarantool-patches
2021-08-19  9:58   ` Serge Petrenko via Tarantool-patches
2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 3/5] box, datetime: datetime comparison for indices Timur Safin via Tarantool-patches
2021-08-19 10:16   ` Serge Petrenko via Tarantool-patches
2021-08-19 11:18   ` UNera via Tarantool-patches
2021-08-19 11:53     ` Safin Timur via Tarantool-patches
2021-08-19 14:47       ` Dmitry E. Oboukhov via Tarantool-patches
2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 4/5] datetime: perf test for datetime parser Timur Safin via Tarantool-patches
2021-08-19 10:19   ` Serge Petrenko via Tarantool-patches
2021-08-19 10:29     ` Safin Timur via Tarantool-patches
2021-08-19 11:11       ` Serge Petrenko via Tarantool-patches
2021-08-19 15:58       ` Vladimir Davydov via Tarantool-patches
2021-08-19  2:56 ` [Tarantool-patches] [PATCH v6 5/5] datetime: changelog for datetime module Timur Safin via Tarantool-patches
2021-08-19 10:20   ` Serge Petrenko via Tarantool-patches

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox