From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 6AC586EC5C; Thu, 15 Jul 2021 11:18:53 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 6AC586EC5C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1626337133; bh=NFaRbKfY3J31ERg+W3JLFWGjMpVramv8ISJ7GsSJosk=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=jKiWcvpiMK5488GP8m2iM9RbTTnNjUuwCgTzfA2NyhEKgGSCY8yQ/PuwzhXXmRhwf Y+0m9pX7fR28qwEOmpz7uut/ioavlXHiQZG2jka6UW1u4g8CuBM4R9oIp8d95Ie1rW lyv0smNN1IaXNFzc7TTxW/QrQRehlx9T6lwxFjL8= Received: from smtp57.i.mail.ru (smtp57.i.mail.ru [217.69.128.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 2D9E86EC55 for ; Thu, 15 Jul 2021 11:18:51 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 2D9E86EC55 Received: by smtp57.i.mail.ru with esmtpa (envelope-from ) id 1m3wa1-00070N-Ma; Thu, 15 Jul 2021 11:18:50 +0300 To: v.shpilevoy@tarantool.org Date: Thu, 15 Jul 2021 11:18:06 +0300 Message-Id: X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-174C08C4: 5188C02AEC42908C481ED7ADC579193296BBA28369E3F2D2713F3D5F7D406D31BCF678C7329BA986 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD941C43E597735A9C351B198F4576AC7B255962EB053FB468C182A05F5380850407EAD183A6605134276146CFCB7726E6EE9FE6FEC9ACC8A067C20010756E8EDEC X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7F65C230EDDCD559EEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006375A514678F9DF65078638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8C7478FFBD1290721E594BAD7A9D6D0DC117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC8C7ADC89C2F0B2A5A471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD182CC0D3CB04F14752D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE140C956E756FBB7AA9539A8B24243104D8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE35FF72824B19451C6302FCEF25BFAB345C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F7900637AD0424077D726551EFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-C1DE0DAB: 8BD88D57C5CADBC8B2710865C386751094C72BDDC9A8ED5CA3B1A56EE2B804F6B226C914C996894645FD9D8A29397D6EFF55FE49A3C2BFCFC8C84E951CD0BE2F296C473AB1E14218EA052B563B0B06C67866D6147AF826D8C7478FFBD1290721E594BAD7A9D6D0DCF972CCD2F8FE1EF1CFC4036BBF6A4EA96EC06C623156B762DA371E42557AFC06296C473AB1E142184421A7F6DAEB7DB2 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D349320586B19AD2C72F4AA3A3713468CF1215661494E557637E6A763B61B4D00C8EB9D39847F8528E01D7E09C32AA3244C3530EEE54CE5BC90933451997F4B71AAE3D93501275E802F83B48618A63566E0 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojbL9S8ysBdXjnYxxC9kpVb5cJjy6CDKCq X-Mailru-Sender: B5B6A6EBBD94DAD8DA84A184D75F19DB2E7B7B20C3E75D0F3B48318E2C468FB0EFE24C4A92C69CBE1EC9E4A2C82A33BC8C24925A86E657CE0C70AEE3C9A96FBAB3D7EE8ED63280BE112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [RFC PATCH 00/13] Initial datetime support X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Timur Safin via Tarantool-patches Reply-To: Timur Safin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" Here is the preliminary RFC version of a patchset, which is implementing datetime lua support in box, their messagepack, yaml, json serialization support, and a set of c and lua tests. * It's heavily influenced by Sci-Lua lua-time module implementation https://github.com/stepelu/lua-time e.g. you could find very similar approach for handling of operations with year and months periods (which should be handled differently than usual seconds, days periods). * But internally it actually uses Christina Hanson' c-dt module https://github.com/chansen/c-dt (though it has been modified slightly for cleaner integration into cmake build process) This is preliminary version of patchset, with unoptimal patches divisions (like you could see patches with rename refactorings), but, due to an amount of code in patchset, and limited time have, I send patchset as RFC today, expecting to receive initial feedback for various parts of a series. Prior plan was to split series into one, which introduces datetime module api, and another one with datetime messagepack serialization implementation. But here they left as part of a same series. Sorry, for invonvenience - I'll reshuffle order according to feedback provided. The problem, which was stopping such simple division - that `datetime.lua` module uses `datetime_to_string()` ffi function for stringization of a datetime object. And this function has been introduced as part of messagepack/yaml/json serialization support. Yes, intra-dependendencies between parts looks suboptimal, and I'm very open to suggestions how to divide it properly. Module API ========== To simplify review of a code in patchset below you could find preliminary documentation of a datetime api module. Eventually we may put it as .rst file in documentation repository (though, personally, I'd prefer to have all modules api committed to the same repo as code, but I digress) Datetime module =============== This module allows to parse date/time stamps, manipulate with them, and generate textual representation. Parsing of date constants will handle all ISO-8601 formats, and deal with some extensions. For internal date representation it uses cdata structures of a form: ```c++ struct t_datetime { int secs; int nsecs; int offset; }; ``` Where: - secs is the (signed) number of seconds since epoch 1970-01-01T00:00Z; - nsecs is number of nanoseconds since beginning of day; - offset is the timezone offset (in minutes); `datetime()` – create datetime object ------------------------------------- Create date/time object using either string literals, or initialization object. When given string literals it behaves like wrapper around parse() method. When there is initialization object it could create date/time object using these attributes below: | **Easy way** | | |--------------|-------------------------------------------------------------------------------------------------| | secs | Seconds since epoch | | nsec | Nanoseconds since midnight | | offset | Time-zone offset in minutes | | **YMD part** | | | year | Year in range \[1..9999\] | | month | Month in range \[1..12\] | | day | Day in month in range \[1..31\] | | **HMS part** | | | hour | Hour in range \[0..23\] | | minute | Minute in range \[0..59\] | | second | Seconds in range \[0..60\]. It allows to have fraction part in which case it goes to nsec field | | tz | Timezone offset (in minutes) for HMS part | Example: ```lua datetime = require `datetime` d = datetime{secs = 0, nsec = 0, offset = 0} d = datetime(‘1970-01-01’) d = datetime{year = 1970, month = 1, day = 1} ``` `delta()` – create time duration object --------------------------------------- TBD `parse()` – parse full ISO-8601 string -------------------------------------- Parse full length date/time literal, which may be in ISO-8601 format of any of extended formats supported by `parse_date()`, `parse_time()` or `parse_timezone()` It deals with date/time string in format `date ([Tt ] time ([ ] time_zone)? )?` Where time or `time_zone` parts may be omitted. Example: ```lua datetime = require `datetime` d = datetime.parse(`1970-01-01`) d = datetime.parse(`1970-01-01T00:00:00Z`) d = datetime.parse(`1970-01-01T02:00:00+02:00`) ``` `parse_date()` – parse ISO-8601 date literal -------------------------------------------- Parse date string literal, return partial date object which has precision of up-to date period of time. A number of standard ISO-8601 formats supported, plus there are some relaxed formats which are of frequently use: | Basic | Extended | | |----------|------------|--------------------------| | 20121224 | 2012-12-24 | Calendar date (ISO 8601) | | 2012359 | 2012-359 | Ordinal date (ISO 8601) | | 2012W521 | 2012-W52-1 | Week date (ISO 8601) | | 2012Q485 | 2012-Q4-85 | Quarter date | `parse_time()` – parse ISO-8601 time literal -------------------------------------------- Parse time string literal, return partial date/time object, which defines time period inside of single date. A number of standard ISO-8601 formats supported, plus there are some relaxed formats which are of frequently use: | Basic | Extended | |-------------------|---------------------| | T12 | N/A | | T1230 | T12:30 | | T123045 | T12:30:45 | | T123045.123456789 | T12:30:45.123456789 | | T123045,123456789 | T12:30:45,123456789 | The time designator T may be omitted. `parse_zone()` – parse ISO-8601 time zone ----------------------------------------- Parse time-zone string literal, return partial date/time object, which defines timezone offset in minutes sing GMT. A number of standard ISO-8601 formats supported, plus there are some relaxed formats which are of frequently use: | Basic | Extended | |-------|----------| | Z | N/A | | ±hh | N/A | | ±hhmm | ±hh:mm | `tostring()` – convert datetime object to string ------------------------------------------------ Return string representation (probably compact if there are some parts missing) of a date-time objects passed `now()` – return current date/time ---------------------------------- `now()` returns local date and time object. It will use local time-zone and nanosecond precision. `strftime()` – convert date object to string using format --------------------------------------------------------- `strftime()` is the FFI wrapper around strftime() function in LIBC. It supports all the same flags which supports strftime() from host OS. See for more details. `asctime()` – convert date object to string using asctime predefined format --------------------------------------------------------------------------- `asctime()` is the FFI wrapper over `asctime_r()` from a host libc. asctime returns string in the form `"Sun Sep 16 01:03:52 1973\\n\\0"` `ctime()` – convert local time to string using ctime() predefined format ------------------------------------------------------------------------ `ctime()` is the FFI wrapper over `ctime_r()` in the host libc. ctime returns string in the form `"Sun Sep 16 01:03:52 1973\\n\\0"` The difference of `ctime()` and `asctime()` is that former is returning local time zone formatted, while the latter will deal with GMT. Examples: ``` tarantool> date = require 'datetime' --- ... tarantool> T = date.now() --- ... tarantool> T --- - 2021-07-14T01:36:48.554105+03:00 ... tarantool> date.asctime(T) --- - 'Tue Jul 13 22:36:48 2021 ' ... tarantool> date.ctime(T) --- - 'Wed Jul 14 01:36:48 2021 ' ... ``` Date attribute accessors ------------------------ | | | |----------------|-----------------------------------------------------------------| | `timestamp` | Calculate timestamp with seconds and nanoseconds parts combined | | `nanoseconds` | Number of nanoseconds in time part | | `microseconds` | Number of microseconds in time part | | `milliseconds` | Number of milliseconds in time part | | `seconds` | Alias to timestamp | | `minutes` | Number of minutes in time part | | `hours` | Number of hours in time part | | `days` | Number of days in time part | ``` tarantool> d = date{year = 1970, month = 1, day = 1, hour = 0, minute = 10, second=10} tarantool> d.secs --- - 610 ... tarantool> d.nsec --- - 0 ... tarantool> d.offset --- - 0 ... tarantool> d.nanoseconds --- - 610000000000 ... tarantool> d.milliseconds --- - 610000 ... tarantool> d.hours --- - 0.16944444444444 ... tarantool> d.minutes --- - 10.166666666667 ... ``` Date arithmetic --------------- TBD https://github.com/tarantool/tarantool/issues/5941 https://github.com/tarantool/tarantool/issues/5946 https://github.com/tarantool/tarantool/tree/tsafin/gh-5941-datetime-V2 Timur Safin (13): build: add Christian Hansen c-dt to the build lua: built-in module datetime test: datetime test test: datetime string formatting box: add messagepack support for datetime lua: positive/negative cases in datetime test lua: asctime and strfime fixed box, lua: renamed t_datetime_tz structure to datetime_t lua: calculated attributes for date lua: tostring formatization in datetime.lua test: allow relaxed date format without tz lua: initial time duration support lua: complete time duration support .gitmodules | 3 + CMakeLists.txt | 8 + cmake/BuildCDT.cmake | 6 + src/CMakeLists.txt | 4 + src/box/field_def.c | 34 +- src/box/field_def.h | 1 + src/box/msgpack.c | 7 +- src/box/tuple_compare.cc | 24 + src/exports.h | 26 + src/lib/core/CMakeLists.txt | 4 +- src/lib/core/datetime.h | 96 ++++ src/lib/core/mp_datetime.c | 232 ++++++++ src/lib/core/mp_extension_types.h | 1 + src/lib/mpstream/mpstream.c | 11 + src/lib/mpstream/mpstream.h | 4 + src/lua/datetime.c | 70 +++ src/lua/datetime.h | 53 ++ src/lua/datetime.lua | 868 ++++++++++++++++++++++++++++++ src/lua/init.c | 6 +- src/lua/msgpack.c | 12 + src/lua/msgpackffi.lua | 8 + src/lua/serializer.c | 4 + src/lua/serializer.h | 2 + src/lua/utils.c | 1 - test/app-tap/datetime.test.lua | 266 +++++++++ test/unit/CMakeLists.txt | 2 + test/unit/datetime.c | 220 ++++++++ test/unit/datetime.result | 358 ++++++++++++ third_party/c-dt | 1 + third_party/lua-cjson/lua_cjson.c | 8 + third_party/lua-yaml/lyaml.cc | 6 +- 31 files changed, 2326 insertions(+), 20 deletions(-) create mode 100644 cmake/BuildCDT.cmake create mode 100644 src/lib/core/datetime.h create mode 100644 src/lib/core/mp_datetime.c create mode 100644 src/lua/datetime.c create mode 100644 src/lua/datetime.h create mode 100644 src/lua/datetime.lua create mode 100755 test/app-tap/datetime.test.lua create mode 100644 test/unit/datetime.c create mode 100644 test/unit/datetime.result create mode 160000 third_party/c-dt -- 2.29.2