From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 88B7E6EC40; Fri, 13 Aug 2021 01:36:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 88B7E6EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1628807812; bh=i0vDSK4ETI7aWlPscBwVnQ41WdjyG+wz489Leo/FHFI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=q2QBNEx+Y5Elet02GSsH04kaAsBxoYicieM47JIJE/9n0wNJS5JEcWfoFLFVx2v+o dFLQ+qczhz0WMc4/m42xVw7KqAG44OEJISqf2RDCc1yOFWqccqsVtVtP3b2yIrciQb F649LBek2C1joqM3eUbKD/XfyLJwOBPiZ8tDDEEk= Received: from smtp52.i.mail.ru (smtp52.i.mail.ru [94.100.177.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 465536EC43 for ; Fri, 13 Aug 2021 01:34:21 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 465536EC43 Received: by smtp52.i.mail.ru with esmtpa (envelope-from ) id 1mEJHH-0003Mm-Mp; Fri, 13 Aug 2021 01:34:20 +0300 To: imun@tarantool.org, v.shpilevoy@tarantool.org Date: Fri, 13 Aug 2021 01:33:44 +0300 Message-Id: <1280270893d4abd584a7e8487a27985c6c5bb1d4.1628805259.git.tsafin@tarantool.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD9F9A2272A1D086A28553D1D5C4B4124EF182A05F53808504021908079EEE6D3019D5F9F96B11E6D32FF1DA0893D7A7EAA3B8CA919F767ED4C X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE77FAFB841CE9024F9EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F790063797024C28700A46CB8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D866E03F2129BCD6644612629AB144FCF4117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18CB629EEF1311BF91D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE41BF15D38FB6CB3A302FCEF25BFAB3454AD6D5ED66289B52698AB9A7B718F8C46E0066C2D8992A16725E5C173C3A84C305525BC79FAC2943BA3038C0950A5D36B5C8C57E37DE458B0BC6067A898B09E46D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE7D699F3A2029486C7731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458F0AFF96BAACF4158235E5A14AD4A4A4625E192CAD1D9E79D94463893BF8742D0871278C1488190DA X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C69415AB31670C86C49C8919565D343F1645AA3DB69AEFDB09C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF246B04B1A760E086699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D341776B9FDE05FBA7A68A644E7A7E56D374025F68ACFDFCFD28F7A21C72FBA2371F9D6572902CE392B1D7E09C32AA3244C0DFC50F2DF0CD086B0AA33B6CB16EB958894E9C85370243E8D5DD81C2BAB7D1D X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2bioj0dLV0c3jbkzqKidxVSNC6g== X-Mailru-Sender: B5B6A6EBBD94DAD86A422A53D9D37A7D23E429C30DF02D8E5F6D821578F0539F8281217E9CDB55391EC9E4A2C82A33BC8C24925A86E657CE0C70AEE3C9A96FBAB3D7EE8ED63280BE112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v4 5/7] box, datetime: datetime comparison for indices X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Timur Safin via Tarantool-patches Reply-To: Timur Safin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" * storage hints implemented for datetime_t values; * proper comparison for indices of datetime type. Part of #5941 Part of #5946 --- src/box/field_def.c | 18 ++++++++ src/box/field_def.h | 3 ++ src/box/memtx_space.c | 3 +- src/box/tuple_compare.cc | 57 ++++++++++++++++++++++++++ src/box/vinyl.c | 3 +- test/engine/datetime.result | 77 +++++++++++++++++++++++++++++++++++ test/engine/datetime.test.lua | 35 ++++++++++++++++ 7 files changed, 192 insertions(+), 4 deletions(-) create mode 100644 test/engine/datetime.result create mode 100644 test/engine/datetime.test.lua diff --git a/src/box/field_def.c b/src/box/field_def.c index 2682a42ee..97033d0bb 100644 --- a/src/box/field_def.c +++ b/src/box/field_def.c @@ -194,3 +194,21 @@ field_type_by_name(const char *name, size_t len) return FIELD_TYPE_ANY; return field_type_MAX; } + +const bool field_type_index_allowed[] = + { + /* [FIELD_TYPE_ANY] = */ false, + /* [FIELD_TYPE_UNSIGNED] = */ true, + /* [FIELD_TYPE_STRING] = */ true, + /* [FIELD_TYPE_NUMBER] = */ true, + /* [FIELD_TYPE_DOUBLE] = */ true, + /* [FIELD_TYPE_INTEGER] = */ true, + /* [FIELD_TYPE_BOOLEAN] = */ true, + /* [FIELD_TYPE_VARBINARY]= */ true, + /* [FIELD_TYPE_SCALAR] = */ true, + /* [FIELD_TYPE_DECIMAL] = */ true, + /* [FIELD_TYPE_UUID] = */ true, + /* [FIELD_TYPE_ARRAY] = */ false, + /* [FIELD_TYPE_MAP] = */ false, + /* [FIELD_TYPE_DATETIME] = */ true, +}; diff --git a/src/box/field_def.h b/src/box/field_def.h index 120b2a93d..bd02418df 100644 --- a/src/box/field_def.h +++ b/src/box/field_def.h @@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[]; extern const struct opt_def field_def_reg[]; extern const struct field_def field_def_default; +/** helper table for checking allowed indices for types */ +extern const bool field_type_index_allowed[]; + /** * @brief Field definition * Contains information about of one tuple field. diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c index b71318d24..1ab16122e 100644 --- a/src/box/memtx_space.c +++ b/src/box/memtx_space.c @@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def) /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; - if (part->type <= FIELD_TYPE_ANY || - part->type >= FIELD_TYPE_ARRAY) { + if (!field_type_index_allowed[part->type]) { diag_set(ClientError, ER_MODIFY_INDEX, index_def->name, space_name(space), tt_sprintf("field type '%s' is not supported", diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc index 9a69f2a72..110017853 100644 --- a/src/box/tuple_compare.cc +++ b/src/box/tuple_compare.cc @@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type, field_b, b_type); case FIELD_TYPE_UUID: return mp_compare_uuid(field_a, field_b); + case FIELD_TYPE_DATETIME: + return mp_compare_datetime(field_a, field_b); default: unreachable(); return 0; @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint, #define HINT_VALUE_DOUBLE_MAX (exp2(HINT_VALUE_BITS - 1) - 1) #define HINT_VALUE_DOUBLE_MIN (-exp2(HINT_VALUE_BITS - 1)) +/** + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds + * into 60 bits of hint value. The idea is to represent wide enough + * years range, and leave the rest of bits occupied from nanoseconds part: + * - 36 bits is enough for time range of [208BC..4147] + * - for nanoseconds there is left 24 bits, which are MSB part of + * 32-bit value + */ +#define HINT_VALUE_SECS_BITS 36 +#define HINT_VALUE_NSEC_BITS (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS) +#define HINT_VALUE_SECS_MAX ((1LL << HINT_VALUE_SECS_BITS) - 1) +#define HINT_VALUE_SECS_MIN (-(1LL << HINT_VALUE_SECS_BITS)) +#define HINT_VALUE_NSEC_SHIFT (sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS) +#define HINT_VALUE_NSEC_MAX ((1ULL << HINT_VALUE_NSEC_BITS) - 1) + /* * HINT_CLASS_BITS should be big enough to store any mp_class value. * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE. @@ -1630,6 +1647,25 @@ hint_uuid_raw(const char *data) return hint_create(MP_CLASS_UUID, val); } +static inline hint_t +hint_datetime(struct datetime *date) +{ + /* + * Use at most HINT_VALUE_SECS_BITS from datetime + * seconds field as a hint value, and at MSB part + * of HINT_VALUE_NSEC_BITS from nanoseconds. + */ + int64_t secs = date->secs; + int32_t nsec = date->nsec; + uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 : + secs - HINT_VALUE_SECS_MIN; + if (val >= HINT_VALUE_SECS_MAX) + val = HINT_VALUE_SECS_MAX; + val <<= HINT_VALUE_NSEC_BITS; + val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX; + return hint_create(MP_CLASS_DATETIME, val); +} + static inline uint64_t hint_str_raw(const char *s, uint32_t len) { @@ -1761,6 +1797,17 @@ field_hint_uuid(const char *field) return hint_uuid_raw(data); } +static inline hint_t +field_hint_datetime(const char *field) +{ + assert(mp_typeof(*field) == MP_EXT); + int8_t ext_type; + uint32_t len = mp_decode_extl(&field, &ext_type); + assert(ext_type == MP_DATETIME); + struct datetime date; + return hint_datetime(datetime_unpack(&field, len, &date)); +} + static inline hint_t field_hint_string(const char *field, struct coll *coll) { @@ -1812,6 +1859,11 @@ field_hint_scalar(const char *field, struct coll *coll) } case MP_UUID: return hint_uuid_raw(field); + case MP_DATETIME: + { + struct datetime date; + return hint_datetime(datetime_unpack(&field, len, &date)); + } default: unreachable(); } @@ -1849,6 +1901,8 @@ field_hint(const char *field, struct coll *coll) return field_hint_decimal(field); case FIELD_TYPE_UUID: return field_hint_uuid(field); + case FIELD_TYPE_DATETIME: + return field_hint_datetime(field); default: unreachable(); } @@ -1963,6 +2017,9 @@ key_def_set_hint_func(struct key_def *def) case FIELD_TYPE_UUID: key_def_set_hint_func(def); break; + case FIELD_TYPE_DATETIME: + key_def_set_hint_func(def); + break; default: /* Invalid key definition. */ def->key_hint = NULL; diff --git a/src/box/vinyl.c b/src/box/vinyl.c index c80b2d99b..360d1fa70 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -662,8 +662,7 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def) /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; - if (part->type <= FIELD_TYPE_ANY || - part->type >= FIELD_TYPE_ARRAY) { + if (!field_type_index_allowed[part->type]) { diag_set(ClientError, ER_MODIFY_INDEX, index_def->name, space_name(space), tt_sprintf("field type '%s' is not supported", diff --git a/test/engine/datetime.result b/test/engine/datetime.result new file mode 100644 index 000000000..848a0aaec --- /dev/null +++ b/test/engine/datetime.result @@ -0,0 +1,77 @@ +-- test-run result file version 2 +env = require('test_run') + | --- + | ... +test_run = env.new() + | --- + | ... +engine = test_run:get_cfg('engine') + | --- + | ... + +date = require('datetime') + | --- + | ... + +_ = box.schema.space.create('T', {engine = engine}) + | --- + | ... +_ = box.space.T:create_index('pk', {parts={1,'datetime'}}) + | --- + | ... + +box.space.T:insert{date('1970-01-01')}\ +box.space.T:insert{date('1970-01-02')}\ +box.space.T:insert{date('1970-01-03')}\ +box.space.T:insert{date('2000-01-01')} + | --- + | ... + +o = box.space.T:select{} + | --- + | ... +assert(tostring(o[1][1]) == '1970-01-01T00:00Z') + | --- + | - true + | ... +assert(tostring(o[2][1]) == '1970-01-02T00:00Z') + | --- + | - true + | ... +assert(tostring(o[3][1]) == '1970-01-03T00:00Z') + | --- + | - true + | ... +assert(tostring(o[4][1]) == '2000-01-01T00:00Z') + | --- + | - true + | ... + +for i = 1,16 do\ + box.space.T:insert{date.now()}\ +end + | --- + | ... + +a = box.space.T:select{} + | --- + | ... +err = {} + | --- + | ... +for i = 1, #a - 1 do\ + if a[i][1] >= a[i+1][1] then\ + table.insert(err, {a[i][1], a[i+1][1]})\ + break\ + end\ +end + | --- + | ... + +err + | --- + | - [] + | ... +box.space.T:drop() + | --- + | ... diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua new file mode 100644 index 000000000..3685e4d4b --- /dev/null +++ b/test/engine/datetime.test.lua @@ -0,0 +1,35 @@ +env = require('test_run') +test_run = env.new() +engine = test_run:get_cfg('engine') + +date = require('datetime') + +_ = box.schema.space.create('T', {engine = engine}) +_ = box.space.T:create_index('pk', {parts={1,'datetime'}}) + +box.space.T:insert{date('1970-01-01')}\ +box.space.T:insert{date('1970-01-02')}\ +box.space.T:insert{date('1970-01-03')}\ +box.space.T:insert{date('2000-01-01')} + +o = box.space.T:select{} +assert(tostring(o[1][1]) == '1970-01-01T00:00Z') +assert(tostring(o[2][1]) == '1970-01-02T00:00Z') +assert(tostring(o[3][1]) == '1970-01-03T00:00Z') +assert(tostring(o[4][1]) == '2000-01-01T00:00Z') + +for i = 1,16 do\ + box.space.T:insert{date.now()}\ +end + +a = box.space.T:select{} +err = {} +for i = 1, #a - 1 do\ + if a[i][1] >= a[i+1][1] then\ + table.insert(err, {a[i][1], a[i+1][1]})\ + break\ + end\ +end + +err +box.space.T:drop() -- 2.29.2