From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 42CF56EC41; Mon, 16 Aug 2021 03:02:28 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 42CF56EC41 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1629072148; bh=i0vDSK4ETI7aWlPscBwVnQ41WdjyG+wz489Leo/FHFI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=md3KBfsUhArorl14ZXpM3ol2sRr0q+cGsA38SvMEpRvSk67AJCUe144vFoHHuB2ZE PuE+wtulSuGoaJOW1eKcwU4F48wwSuXa+slGvzgqp+l7XYL16LOtn9rvwkcVHwDCSf KccSIfiUbxqN58sCIUEs5q8DflNmDLUOeFrnXxmE= Received: from smtp41.i.mail.ru (smtp41.i.mail.ru [94.100.177.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 7C1E96EC5D for ; Mon, 16 Aug 2021 03:00:01 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 7C1E96EC5D Received: by smtp41.i.mail.ru with esmtpa (envelope-from ) id 1mFQ2q-00047O-1g; Mon, 16 Aug 2021 03:00:00 +0300 To: v.shpilevoy@tarantool.org, imun@tarantool.org, imeevma@tarantool.org, tarantool-patches@dev.tarantool.org Date: Mon, 16 Aug 2021 02:59:39 +0300 Message-Id: <906cfac8b66b70eb982b5c7fe4878da39712a305.1629071531.git.tsafin@tarantool.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD91BCCB18F2C129F87F36E61E9E4584E9D182A05F538085040E02464DD3AD5F03B190F5FF0C89167B83501EB2A1BD86F7F73AD4E47C1D7F094 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7A20935EE237A17ECEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006377CB294AC37272EBD8638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D86FBD6C639F2F5C1C226EFBBFFEA4C108117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18C26CFBAC0749D213D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE599709FD55CB46A6F206494F22AA87D6D8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE362B3BD3CC35DA588BA3038C0950A5D36B5C8C57E37DE458B0BC6067A898B09E46D1867E19FE14079C09775C1D3CA48CF3D321E7403792E342EB15956EA79C166A417C69337E82CC275ECD9A6C639B01B78DA827A17800CE76515C59FC18CEA6D731C566533BA786AA5CC5B56E945C8DA X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458F0AFF96BAACF4158235E5A14AD4A4A4625E192CAD1D9E79D0B18DC6AC13D9A1C5B2128469A12B8DA X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C5B73F950BC6E7FFB0694A92D620630DFE89505B4B9A104809C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EFE37876E7723AB534DC48ACC2A39D04F89CDFB48F4795C241BDAD6C7F3747799A X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D3442AF01057B6BF997D561BF2B2949F9E102709A5A41BDAF4F29F1C92182DBB8F95BCC6495F4AA0D501D7E09C32AA3244CA6EECF654CB9902A318FAB2EAD402C117C0C08F7987826B98D5DD81C2BAB7D1D X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojVT2don9h9KpmXNX0nFb0TQ== X-Mailru-Sender: B5B6A6EBBD94DAD833E4B9A85F720031806AC487ADECA3543CCB2058E4AA6C26F7945288957490F11EC9E4A2C82A33BC8C24925A86E657CE0C70AEE3C9A96FBAB3D7EE8ED63280BE112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v5 5/8] box, datetime: datetime comparison for indices X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Timur Safin via Tarantool-patches Reply-To: Timur Safin Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" * storage hints implemented for datetime_t values; * proper comparison for indices of datetime type. Part of #5941 Part of #5946 --- src/box/field_def.c | 18 ++++++++ src/box/field_def.h | 3 ++ src/box/memtx_space.c | 3 +- src/box/tuple_compare.cc | 57 ++++++++++++++++++++++++++ src/box/vinyl.c | 3 +- test/engine/datetime.result | 77 +++++++++++++++++++++++++++++++++++ test/engine/datetime.test.lua | 35 ++++++++++++++++ 7 files changed, 192 insertions(+), 4 deletions(-) create mode 100644 test/engine/datetime.result create mode 100644 test/engine/datetime.test.lua diff --git a/src/box/field_def.c b/src/box/field_def.c index 2682a42ee..97033d0bb 100644 --- a/src/box/field_def.c +++ b/src/box/field_def.c @@ -194,3 +194,21 @@ field_type_by_name(const char *name, size_t len) return FIELD_TYPE_ANY; return field_type_MAX; } + +const bool field_type_index_allowed[] = + { + /* [FIELD_TYPE_ANY] = */ false, + /* [FIELD_TYPE_UNSIGNED] = */ true, + /* [FIELD_TYPE_STRING] = */ true, + /* [FIELD_TYPE_NUMBER] = */ true, + /* [FIELD_TYPE_DOUBLE] = */ true, + /* [FIELD_TYPE_INTEGER] = */ true, + /* [FIELD_TYPE_BOOLEAN] = */ true, + /* [FIELD_TYPE_VARBINARY]= */ true, + /* [FIELD_TYPE_SCALAR] = */ true, + /* [FIELD_TYPE_DECIMAL] = */ true, + /* [FIELD_TYPE_UUID] = */ true, + /* [FIELD_TYPE_ARRAY] = */ false, + /* [FIELD_TYPE_MAP] = */ false, + /* [FIELD_TYPE_DATETIME] = */ true, +}; diff --git a/src/box/field_def.h b/src/box/field_def.h index 120b2a93d..bd02418df 100644 --- a/src/box/field_def.h +++ b/src/box/field_def.h @@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[]; extern const struct opt_def field_def_reg[]; extern const struct field_def field_def_default; +/** helper table for checking allowed indices for types */ +extern const bool field_type_index_allowed[]; + /** * @brief Field definition * Contains information about of one tuple field. diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c index b71318d24..1ab16122e 100644 --- a/src/box/memtx_space.c +++ b/src/box/memtx_space.c @@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def) /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; - if (part->type <= FIELD_TYPE_ANY || - part->type >= FIELD_TYPE_ARRAY) { + if (!field_type_index_allowed[part->type]) { diag_set(ClientError, ER_MODIFY_INDEX, index_def->name, space_name(space), tt_sprintf("field type '%s' is not supported", diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc index 9a69f2a72..110017853 100644 --- a/src/box/tuple_compare.cc +++ b/src/box/tuple_compare.cc @@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type, field_b, b_type); case FIELD_TYPE_UUID: return mp_compare_uuid(field_a, field_b); + case FIELD_TYPE_DATETIME: + return mp_compare_datetime(field_a, field_b); default: unreachable(); return 0; @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint, #define HINT_VALUE_DOUBLE_MAX (exp2(HINT_VALUE_BITS - 1) - 1) #define HINT_VALUE_DOUBLE_MIN (-exp2(HINT_VALUE_BITS - 1)) +/** + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds + * into 60 bits of hint value. The idea is to represent wide enough + * years range, and leave the rest of bits occupied from nanoseconds part: + * - 36 bits is enough for time range of [208BC..4147] + * - for nanoseconds there is left 24 bits, which are MSB part of + * 32-bit value + */ +#define HINT_VALUE_SECS_BITS 36 +#define HINT_VALUE_NSEC_BITS (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS) +#define HINT_VALUE_SECS_MAX ((1LL << HINT_VALUE_SECS_BITS) - 1) +#define HINT_VALUE_SECS_MIN (-(1LL << HINT_VALUE_SECS_BITS)) +#define HINT_VALUE_NSEC_SHIFT (sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS) +#define HINT_VALUE_NSEC_MAX ((1ULL << HINT_VALUE_NSEC_BITS) - 1) + /* * HINT_CLASS_BITS should be big enough to store any mp_class value. * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE. @@ -1630,6 +1647,25 @@ hint_uuid_raw(const char *data) return hint_create(MP_CLASS_UUID, val); } +static inline hint_t +hint_datetime(struct datetime *date) +{ + /* + * Use at most HINT_VALUE_SECS_BITS from datetime + * seconds field as a hint value, and at MSB part + * of HINT_VALUE_NSEC_BITS from nanoseconds. + */ + int64_t secs = date->secs; + int32_t nsec = date->nsec; + uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 : + secs - HINT_VALUE_SECS_MIN; + if (val >= HINT_VALUE_SECS_MAX) + val = HINT_VALUE_SECS_MAX; + val <<= HINT_VALUE_NSEC_BITS; + val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX; + return hint_create(MP_CLASS_DATETIME, val); +} + static inline uint64_t hint_str_raw(const char *s, uint32_t len) { @@ -1761,6 +1797,17 @@ field_hint_uuid(const char *field) return hint_uuid_raw(data); } +static inline hint_t +field_hint_datetime(const char *field) +{ + assert(mp_typeof(*field) == MP_EXT); + int8_t ext_type; + uint32_t len = mp_decode_extl(&field, &ext_type); + assert(ext_type == MP_DATETIME); + struct datetime date; + return hint_datetime(datetime_unpack(&field, len, &date)); +} + static inline hint_t field_hint_string(const char *field, struct coll *coll) { @@ -1812,6 +1859,11 @@ field_hint_scalar(const char *field, struct coll *coll) } case MP_UUID: return hint_uuid_raw(field); + case MP_DATETIME: + { + struct datetime date; + return hint_datetime(datetime_unpack(&field, len, &date)); + } default: unreachable(); } @@ -1849,6 +1901,8 @@ field_hint(const char *field, struct coll *coll) return field_hint_decimal(field); case FIELD_TYPE_UUID: return field_hint_uuid(field); + case FIELD_TYPE_DATETIME: + return field_hint_datetime(field); default: unreachable(); } @@ -1963,6 +2017,9 @@ key_def_set_hint_func(struct key_def *def) case FIELD_TYPE_UUID: key_def_set_hint_func(def); break; + case FIELD_TYPE_DATETIME: + key_def_set_hint_func(def); + break; default: /* Invalid key definition. */ def->key_hint = NULL; diff --git a/src/box/vinyl.c b/src/box/vinyl.c index c80b2d99b..360d1fa70 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -662,8 +662,7 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def) /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; - if (part->type <= FIELD_TYPE_ANY || - part->type >= FIELD_TYPE_ARRAY) { + if (!field_type_index_allowed[part->type]) { diag_set(ClientError, ER_MODIFY_INDEX, index_def->name, space_name(space), tt_sprintf("field type '%s' is not supported", diff --git a/test/engine/datetime.result b/test/engine/datetime.result new file mode 100644 index 000000000..848a0aaec --- /dev/null +++ b/test/engine/datetime.result @@ -0,0 +1,77 @@ +-- test-run result file version 2 +env = require('test_run') + | --- + | ... +test_run = env.new() + | --- + | ... +engine = test_run:get_cfg('engine') + | --- + | ... + +date = require('datetime') + | --- + | ... + +_ = box.schema.space.create('T', {engine = engine}) + | --- + | ... +_ = box.space.T:create_index('pk', {parts={1,'datetime'}}) + | --- + | ... + +box.space.T:insert{date('1970-01-01')}\ +box.space.T:insert{date('1970-01-02')}\ +box.space.T:insert{date('1970-01-03')}\ +box.space.T:insert{date('2000-01-01')} + | --- + | ... + +o = box.space.T:select{} + | --- + | ... +assert(tostring(o[1][1]) == '1970-01-01T00:00Z') + | --- + | - true + | ... +assert(tostring(o[2][1]) == '1970-01-02T00:00Z') + | --- + | - true + | ... +assert(tostring(o[3][1]) == '1970-01-03T00:00Z') + | --- + | - true + | ... +assert(tostring(o[4][1]) == '2000-01-01T00:00Z') + | --- + | - true + | ... + +for i = 1,16 do\ + box.space.T:insert{date.now()}\ +end + | --- + | ... + +a = box.space.T:select{} + | --- + | ... +err = {} + | --- + | ... +for i = 1, #a - 1 do\ + if a[i][1] >= a[i+1][1] then\ + table.insert(err, {a[i][1], a[i+1][1]})\ + break\ + end\ +end + | --- + | ... + +err + | --- + | - [] + | ... +box.space.T:drop() + | --- + | ... diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua new file mode 100644 index 000000000..3685e4d4b --- /dev/null +++ b/test/engine/datetime.test.lua @@ -0,0 +1,35 @@ +env = require('test_run') +test_run = env.new() +engine = test_run:get_cfg('engine') + +date = require('datetime') + +_ = box.schema.space.create('T', {engine = engine}) +_ = box.space.T:create_index('pk', {parts={1,'datetime'}}) + +box.space.T:insert{date('1970-01-01')}\ +box.space.T:insert{date('1970-01-02')}\ +box.space.T:insert{date('1970-01-03')}\ +box.space.T:insert{date('2000-01-01')} + +o = box.space.T:select{} +assert(tostring(o[1][1]) == '1970-01-01T00:00Z') +assert(tostring(o[2][1]) == '1970-01-02T00:00Z') +assert(tostring(o[3][1]) == '1970-01-03T00:00Z') +assert(tostring(o[4][1]) == '2000-01-01T00:00Z') + +for i = 1,16 do\ + box.space.T:insert{date.now()}\ +end + +a = box.space.T:select{} +err = {} +for i = 1, #a - 1 do\ + if a[i][1] >= a[i+1][1] then\ + table.insert(err, {a[i][1], a[i+1][1]})\ + break\ + end\ +end + +err +box.space.T:drop() -- 2.29.2