From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id B2F4F6EC5F; Mon, 2 Aug 2021 03:44:36 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org B2F4F6EC5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1627865076; bh=Ec0K2V6XU3LD7GEgN1wnpHGdQNw89FYfHTQIX8Y3X5k=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=NXqFJXBKKanI674B6XgfP3t52kwzPiXg2BkVe1Cmm7JnA4BfpVu8OYU3hio0av9wG ITJ1G419UNl3MB+i2Jrepv8I1VQZaUYndNKrJwFO4Sr3lD5kmvFTTj0RJ9wVJmfvCn h0hQ/m/70XF6rGosoRk75dgXsrvbhBcsGGp3UrAU= Received: from smtp58.i.mail.ru (smtp58.i.mail.ru [217.69.128.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 954846E471 for ; Mon, 2 Aug 2021 03:41:35 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 954846E471 Received: by smtp58.i.mail.ru with esmtpa (envelope-from ) id 1mAM1N-0000PY-TZ; Mon, 02 Aug 2021 03:41:34 +0300 To: v.shpilevoy@tarantool.org, olegrok@tarantool.org Date: Mon, 2 Aug 2021 03:41:02 +0300 Message-Id: <726aea88a704e2eb11257d5de49fd03818a573ed.1627864075.git.tsafin@tarantool.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-4EC0790: 10 X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD941C43E597735A9C351B198F4576AC7B21928AAE70459C21B182A05F5380850405A497D00F9B0E64DB2B73178B2900726E934F3C5D07801E73D433D204A322D29 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE7CB1634DB9A2F7B99EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AE01C1A20EF0A1348638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8C0E6E5172E6A3ABE7D967C268DB8E2CC117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18C26CFBAC0749D213D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE599709FD55CB46A649AF716F719AB83ED8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE3CE135D2742255B356E0066C2D8992A16C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F7900637B8F435DEDE9E76EBEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B505E13A399CA3FB2FCD7D9A2229451571F1 X-C1DE0DAB: C20DE7B7AB408E4181F030C43753B8186998911F362727C414F749A5E30D975C85811CA530B504917735B8679B9C7FEC0A801D113C7BA39E9C2B6934AE262D3EE7EAB7254005DCED7532B743992DF240BDC6A1CF3F042BAD6DF99611D93F60EF03D88DA7F5FEDC25699F904B3F4130E343918A1A30D5E7FCCB5012B2E24CD356 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34411AA3F52B5778B676C4AAA319ACC79709F69704FEEEF927CB91A4BBB46A3210496D39CD7FAB93E21D7E09C32AA3244CE89FB90CD20007B3A00BD289013A261A63871F383B54D9B38D5DD81C2BAB7D1D X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojMCfuYI4PredX08e5h1KCDw== X-Mailru-Sender: B5B6A6EBBD94DAD8315E1362B7B20B88741BE662BE94462380DADE4B61A702D0F33524C0AB8586555C2808D6142752370A8ED71B308007E3DC85537438B7E1A423D748DE48713E689437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: [Tarantool-patches] [PATCH v3 6/9] box, datetime: datetime comparison for indices X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Timur Safin via Tarantool-patches Reply-To: Timur Safin Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" * storage hints implemented for datetime_t values; * proper comparison for indices of datetime type. Part of #5941 Part of #5946 --- src/box/field_def.c | 18 ++++++++ src/box/field_def.h | 3 ++ src/box/memtx_space.c | 3 +- src/box/tuple_compare.cc | 56 +++++++++++++++++++++++++ src/box/vinyl.c | 3 +- test/engine/datetime.result | 77 +++++++++++++++++++++++++++++++++++ test/engine/datetime.test.lua | 35 ++++++++++++++++ 7 files changed, 191 insertions(+), 4 deletions(-) create mode 100644 test/engine/datetime.result create mode 100644 test/engine/datetime.test.lua diff --git a/src/box/field_def.c b/src/box/field_def.c index 6964e3e9f..aaf5f9cff 100644 --- a/src/box/field_def.c +++ b/src/box/field_def.c @@ -193,3 +193,21 @@ field_type_by_name(const char *name, size_t len) return FIELD_TYPE_ANY; return field_type_MAX; } + +const bool field_type_index_allowed[] = + { + /* [FIELD_TYPE_ANY] = */ false, + /* [FIELD_TYPE_UNSIGNED] = */ true, + /* [FIELD_TYPE_STRING] = */ true, + /* [FIELD_TYPE_NUMBER] = */ true, + /* [FIELD_TYPE_DOUBLE] = */ true, + /* [FIELD_TYPE_INTEGER] = */ true, + /* [FIELD_TYPE_BOOLEAN] = */ true, + /* [FIELD_TYPE_VARBINARY]= */ true, + /* [FIELD_TYPE_SCALAR] = */ true, + /* [FIELD_TYPE_DECIMAL] = */ true, + /* [FIELD_TYPE_UUID] = */ true, + /* [FIELD_TYPE_ARRAY] = */ false, + /* [FIELD_TYPE_MAP] = */ false, + /* [FIELD_TYPE_DATETIME] = */ true, +}; diff --git a/src/box/field_def.h b/src/box/field_def.h index 120b2a93d..bd02418df 100644 --- a/src/box/field_def.h +++ b/src/box/field_def.h @@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[]; extern const struct opt_def field_def_reg[]; extern const struct field_def field_def_default; +/** helper table for checking allowed indices for types */ +extern const bool field_type_index_allowed[]; + /** * @brief Field definition * Contains information about of one tuple field. diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c index b71318d24..1ab16122e 100644 --- a/src/box/memtx_space.c +++ b/src/box/memtx_space.c @@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, struct index_def *index_def) /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; - if (part->type <= FIELD_TYPE_ANY || - part->type >= FIELD_TYPE_ARRAY) { + if (!field_type_index_allowed[part->type]) { diag_set(ClientError, ER_MODIFY_INDEX, index_def->name, space_name(space), tt_sprintf("field type '%s' is not supported", diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc index 48c833643..f733b9f01 100644 --- a/src/box/tuple_compare.cc +++ b/src/box/tuple_compare.cc @@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type, field_b, b_type); case FIELD_TYPE_UUID: return mp_compare_uuid(field_a, field_b); + case FIELD_TYPE_DATETIME: + return mp_compare_datetime(field_a, field_b); default: unreachable(); return 0; @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint, #define HINT_VALUE_DOUBLE_MAX (exp2(HINT_VALUE_BITS - 1) - 1) #define HINT_VALUE_DOUBLE_MIN (-exp2(HINT_VALUE_BITS - 1)) +/** + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds + * into 60 bits of hint value. The idea is to represent wide enough + * years range, and leave the rest of bits occupied from nanoseconds part: + * - 36 bits is enough for time range of [208BC..4147] + * - for nanoseconds there is left 24 bits, which are MSB part of + * 32-bit value + */ +#define HINT_VALUE_SECS_BITS 36 +#define HINT_VALUE_NSEC_BITS (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS) +#define HINT_VALUE_SECS_MAX ((1LL << HINT_VALUE_SECS_BITS) - 1) +#define HINT_VALUE_SECS_MIN (-(1LL << HINT_VALUE_SECS_BITS)) +#define HINT_VALUE_NSEC_SHIFT (sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS) +#define HINT_VALUE_NSEC_MAX ((1ULL << HINT_VALUE_NSEC_BITS) - 1) + /* * HINT_CLASS_BITS should be big enough to store any mp_class value. * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE. @@ -1630,6 +1647,24 @@ hint_uuid_raw(const char *data) return hint_create(MP_CLASS_UUID, val); } +static inline hint_t +hint_datetime(struct datetime_t *date) +{ + /* + * Use at most HINT_VALUE_SECS_BITS from datetime + * seconds field as a hint value, and at MSB part + * of HINT_VALUE_NSEC_BITS from nanoseconds. + */ + int64_t secs = date->secs; + int32_t nsec = date->nsec; + uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 : + secs - HINT_VALUE_SECS_MIN; + val &= HINT_VALUE_SECS_MAX; + val <<= HINT_VALUE_NSEC_BITS; + val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX; + return hint_create(MP_CLASS_DATETIME, val); +} + static inline uint64_t hint_str_raw(const char *s, uint32_t len) { @@ -1761,6 +1796,17 @@ field_hint_uuid(const char *field) return hint_uuid_raw(data); } +static inline hint_t +field_hint_datetime(const char *field) +{ + assert(mp_typeof(*field) == MP_EXT); + int8_t ext_type; + uint32_t len = mp_decode_extl(&field, &ext_type); + assert(ext_type == MP_DATETIME); + struct datetime_t date; + return hint_datetime(datetime_unpack(&field, len, &date)); +} + static inline hint_t field_hint_string(const char *field, struct coll *coll) { @@ -1812,6 +1858,11 @@ field_hint_scalar(const char *field, struct coll *coll) } case MP_UUID: return hint_uuid_raw(field); + case MP_DATETIME: + { + struct datetime_t date; + return hint_datetime(datetime_unpack(&field, len, &date)); + } default: unreachable(); } @@ -1849,6 +1900,8 @@ field_hint(const char *field, struct coll *coll) return field_hint_decimal(field); case FIELD_TYPE_UUID: return field_hint_uuid(field); + case FIELD_TYPE_DATETIME: + return field_hint_datetime(field); default: unreachable(); } @@ -1963,6 +2016,9 @@ key_def_set_hint_func(struct key_def *def) case FIELD_TYPE_UUID: key_def_set_hint_func(def); break; + case FIELD_TYPE_DATETIME: + key_def_set_hint_func(def); + break; default: /* Invalid key definition. */ def->key_hint = NULL; diff --git a/src/box/vinyl.c b/src/box/vinyl.c index c80b2d99b..360d1fa70 100644 --- a/src/box/vinyl.c +++ b/src/box/vinyl.c @@ -662,8 +662,7 @@ vinyl_space_check_index_def(struct space *space, struct index_def *index_def) /* Check that there are no ANY, ARRAY, MAP parts */ for (uint32_t i = 0; i < key_def->part_count; i++) { struct key_part *part = &key_def->parts[i]; - if (part->type <= FIELD_TYPE_ANY || - part->type >= FIELD_TYPE_ARRAY) { + if (!field_type_index_allowed[part->type]) { diag_set(ClientError, ER_MODIFY_INDEX, index_def->name, space_name(space), tt_sprintf("field type '%s' is not supported", diff --git a/test/engine/datetime.result b/test/engine/datetime.result new file mode 100644 index 000000000..848a0aaec --- /dev/null +++ b/test/engine/datetime.result @@ -0,0 +1,77 @@ +-- test-run result file version 2 +env = require('test_run') + | --- + | ... +test_run = env.new() + | --- + | ... +engine = test_run:get_cfg('engine') + | --- + | ... + +date = require('datetime') + | --- + | ... + +_ = box.schema.space.create('T', {engine = engine}) + | --- + | ... +_ = box.space.T:create_index('pk', {parts={1,'datetime'}}) + | --- + | ... + +box.space.T:insert{date('1970-01-01')}\ +box.space.T:insert{date('1970-01-02')}\ +box.space.T:insert{date('1970-01-03')}\ +box.space.T:insert{date('2000-01-01')} + | --- + | ... + +o = box.space.T:select{} + | --- + | ... +assert(tostring(o[1][1]) == '1970-01-01T00:00Z') + | --- + | - true + | ... +assert(tostring(o[2][1]) == '1970-01-02T00:00Z') + | --- + | - true + | ... +assert(tostring(o[3][1]) == '1970-01-03T00:00Z') + | --- + | - true + | ... +assert(tostring(o[4][1]) == '2000-01-01T00:00Z') + | --- + | - true + | ... + +for i = 1,16 do\ + box.space.T:insert{date.now()}\ +end + | --- + | ... + +a = box.space.T:select{} + | --- + | ... +err = {} + | --- + | ... +for i = 1, #a - 1 do\ + if a[i][1] >= a[i+1][1] then\ + table.insert(err, {a[i][1], a[i+1][1]})\ + break\ + end\ +end + | --- + | ... + +err + | --- + | - [] + | ... +box.space.T:drop() + | --- + | ... diff --git a/test/engine/datetime.test.lua b/test/engine/datetime.test.lua new file mode 100644 index 000000000..3685e4d4b --- /dev/null +++ b/test/engine/datetime.test.lua @@ -0,0 +1,35 @@ +env = require('test_run') +test_run = env.new() +engine = test_run:get_cfg('engine') + +date = require('datetime') + +_ = box.schema.space.create('T', {engine = engine}) +_ = box.space.T:create_index('pk', {parts={1,'datetime'}}) + +box.space.T:insert{date('1970-01-01')}\ +box.space.T:insert{date('1970-01-02')}\ +box.space.T:insert{date('1970-01-03')}\ +box.space.T:insert{date('2000-01-01')} + +o = box.space.T:select{} +assert(tostring(o[1][1]) == '1970-01-01T00:00Z') +assert(tostring(o[2][1]) == '1970-01-02T00:00Z') +assert(tostring(o[3][1]) == '1970-01-03T00:00Z') +assert(tostring(o[4][1]) == '2000-01-01T00:00Z') + +for i = 1,16 do\ + box.space.T:insert{date.now()}\ +end + +a = box.space.T:select{} +err = {} +for i = 1, #a - 1 do\ + if a[i][1] >= a[i+1][1] then\ + table.insert(err, {a[i][1], a[i+1][1]})\ + break\ + end\ +end + +err +box.space.T:drop() -- 2.29.2