From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 43BFA6EC40; Wed, 18 Aug 2021 02:43:50 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 43BFA6EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1629243830; bh=5vsJR1F40oYZhv8x0RJB1tPKDzwmm2V/NE40ykWVNkk=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=oj5eJ33jLAQfc7pcltjwvlua4C6ZhfTy4fTRGLOMyCxbFmjkziamwuOXlZugWCyj3 WuhaqFYZBHwvoDHXhABqZh2fxFt3DKem4DsqkYVbfak8QRZbfDBKmNNsqVoFq1ZblK yyFApy78p8nYuKABUp6JjJh6rBlRBE7afD2B+8z8= Received: from smtp57.i.mail.ru (smtp57.i.mail.ru [217.69.128.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 4BCF56EC40 for ; Wed, 18 Aug 2021 02:43:48 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 4BCF56EC40 Received: by smtp57.i.mail.ru with esmtpa (envelope-from ) id 1mG8kE-0002Vy-Oz; Wed, 18 Aug 2021 02:43:47 +0300 To: Serge Petrenko References: <906cfac8b66b70eb982b5c7fe4878da39712a305.1629071531.git.tsafin@tarantool.org> <7baf64ec-6033-8c96-c57e-eb3394ec0b07@tarantool.org> Message-ID: <92ef16db-d5a0-6773-97c6-24a4ba9007cc@tarantool.org> Date: Wed, 18 Aug 2021 02:43:34 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <7baf64ec-6033-8c96-c57e-eb3394ec0b07@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-7564579A: 646B95376F6C166E X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD9BCE6B93DE0C6C3914462CDB1732D383C182A05F5380850400BB45F072E694FC69FFE6DF82BFF91EB4D91CEA5C344F3AB70B717F053425DA1 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE78044725E6B9C7F2CEA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F79006377A58729D825217838638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D8F7D6389E1796D32507608786F5166684117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18618001F51B5FD3F9D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE7B96B19DC409332186A01F665E7D37C5D8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE367F1C1C3ABB44F3A6E0066C2D8992A16C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F790063788B3B24285A3CD0EEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458F0AFF96BAACF4158235E5A14AD4A4A4625E192CAD1D9E79D0B18DC6AC13D9A1C521ED22DADF76659 X-C1DE0DAB: 0D63561A33F958A53334595EE303A04AFA44A184AD0C84C0F03AD2BE6A146300D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA757E10A58996CBD514410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34A533D6DF7731C664A522A68D574DD53503E57524711FBDD8F0682658F2AA307C968FE34142DB38E51D7E09C32AA3244C7A45C9488952B99A551DB44E15D66C3E858039643087248083B48618A63566E0 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojuRQ/H5n28tqBRw56lj6dTg== X-Mailru-Sender: B5B6A6EBBD94DAD8C64AB2DDE1BF89F4A38296DB289D66244E1729E2A08894FE1DB6227F693E0D685C2808D6142752370A8ED71B308007E3DC85537438B7E1A423D748DE48713E689437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v5 5/8] box, datetime: datetime comparison for indices X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Safin Timur via Tarantool-patches Reply-To: Safin Timur Cc: tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" On 17.08.2021 15:16, Serge Petrenko wrote: > > > 16.08.2021 02:59, Timur Safin via Tarantool-patches пишет: >> * storage hints implemented for datetime_t values; >> * proper comparison for indices of datetime type. >> >> Part of #5941 >> Part of #5946 > > > Please, add a docbot request stating that it's now possible to store > datetime values in spaces and create indexed datetime fields. Will use something like that: @TarantoolBot document Title: Storage support for datetime values It's now possible to store datetime values in spaces and create indexed datetime fields. Please refer to https://github.com/tarantool/tarantool/discussions/6244 for more detailed description of a storage schema. > > >> --- >>   src/box/field_def.c           | 18 ++++++++ >>   src/box/field_def.h           |  3 ++ >>   src/box/memtx_space.c         |  3 +- >>   src/box/tuple_compare.cc      | 57 ++++++++++++++++++++++++++ >>   src/box/vinyl.c               |  3 +- >>   test/engine/datetime.result   | 77 +++++++++++++++++++++++++++++++++++ >>   test/engine/datetime.test.lua | 35 ++++++++++++++++ >>   7 files changed, 192 insertions(+), 4 deletions(-) >>   create mode 100644 test/engine/datetime.result >>   create mode 100644 test/engine/datetime.test.lua >> >> diff --git a/src/box/field_def.c b/src/box/field_def.c >> index 2682a42ee..97033d0bb 100644 >> --- a/src/box/field_def.c >> +++ b/src/box/field_def.c >> @@ -194,3 +194,21 @@ field_type_by_name(const char *name, size_t len) >>           return FIELD_TYPE_ANY; >>       return field_type_MAX; >>   } >> + >> +const bool field_type_index_allowed[] = >> +    { >> +    /* [FIELD_TYPE_ANY]      = */ false, >> +    /* [FIELD_TYPE_UNSIGNED] = */ true, >> +    /* [FIELD_TYPE_STRING]   = */ true, >> +    /* [FIELD_TYPE_NUMBER]   = */ true, >> +    /* [FIELD_TYPE_DOUBLE]   = */ true, >> +    /* [FIELD_TYPE_INTEGER]  = */ true, >> +    /* [FIELD_TYPE_BOOLEAN]  = */ true, >> +    /* [FIELD_TYPE_VARBINARY]= */ true, >> +    /* [FIELD_TYPE_SCALAR]   = */ true, >> +    /* [FIELD_TYPE_DECIMAL]  = */ true, >> +    /* [FIELD_TYPE_UUID]     = */ true, >> +    /* [FIELD_TYPE_ARRAY]    = */ false, >> +    /* [FIELD_TYPE_MAP]      = */ false, >> +    /* [FIELD_TYPE_DATETIME] = */ true, >> +}; > > > You wouldn't need that array if you moved > FIELD_TYPE_DATETIME above FIELD_TYPE_ARRAY > in the previous commit. > > Please, do so. Yes, will change order and also move all field support code to this patch (as Vova recommends). > > >> diff --git a/src/box/field_def.h b/src/box/field_def.h >> index 120b2a93d..bd02418df 100644 >> --- a/src/box/field_def.h >> +++ b/src/box/field_def.h >> @@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[]; >>   extern const struct opt_def field_def_reg[]; >>   extern const struct field_def field_def_default; >> +/** helper table for checking allowed indices for types */ >> +extern const bool field_type_index_allowed[]; >> + >>   /** >>    * @brief Field definition >>    * Contains information about of one tuple field. >> diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c >> index b71318d24..1ab16122e 100644 >> --- a/src/box/memtx_space.c >> +++ b/src/box/memtx_space.c >> @@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, >> struct index_def *index_def) >>       /* Check that there are no ANY, ARRAY, MAP parts */ >>       for (uint32_t i = 0; i < key_def->part_count; i++) { >>           struct key_part *part = &key_def->parts[i]; >> -        if (part->type <= FIELD_TYPE_ANY || >> -            part->type >= FIELD_TYPE_ARRAY) { >> +        if (!field_type_index_allowed[part->type]) { >>               diag_set(ClientError, ER_MODIFY_INDEX, >>                    index_def->name, space_name(space), >>                    tt_sprintf("field type '%s' is not supported", >> diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc >> index 9a69f2a72..110017853 100644 >> --- a/src/box/tuple_compare.cc >> +++ b/src/box/tuple_compare.cc >> @@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char *field_a, >> enum mp_type a_type, >>                              field_b, b_type); >>       case FIELD_TYPE_UUID: >>           return mp_compare_uuid(field_a, field_b); >> +    case FIELD_TYPE_DATETIME: >> +        return mp_compare_datetime(field_a, field_b); >>       default: >>           unreachable(); >>           return 0; >> @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple >> *tuple, hint_t tuple_hint, >>   #define HINT_VALUE_DOUBLE_MAX    (exp2(HINT_VALUE_BITS - 1) - 1) >>   #define HINT_VALUE_DOUBLE_MIN    (-exp2(HINT_VALUE_BITS - 1)) >> +/** >> + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds >> + * into 60 bits of hint value. The idea is to represent wide enough >> + * years range, and leave the rest of bits occupied from nanoseconds >> part: >> + * - 36 bits is enough for time range of [208BC..4147] >> + * - for nanoseconds there is left 24 bits, which are MSB part of >> + *   32-bit value >> + */ >> +#define HINT_VALUE_SECS_BITS    36 >> +#define HINT_VALUE_NSEC_BITS    (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS) >> +#define HINT_VALUE_SECS_MAX    ((1LL << HINT_VALUE_SECS_BITS) - 1) > > Am I missing something? > n bits may store values from (-2^(n-1)) to 2^(n-1)-1 > > should be (1LL << (HINT_VALUE_SECS_BITS -1))  - 1 ? > > >> +#define HINT_VALUE_SECS_MIN    (-(1LL << HINT_VALUE_SECS_BITS)) > > > > > should be > > #define HINT_VALUE_SECS_MIN    (-(1LL << (HINT_VALUE_SECS_BITS - 1))) > > ? > Yes, my definition made sense only when used as a mask (in prior version of a code). Thus did not take into consideration a sign bit. You absolutely correct that if seconds are signed then we have lesser number of bits, and your definitions of HINT_VALUE_SECS_MAX/HINT_VALUE_SECS_MIN should be used. > >> +#define HINT_VALUE_NSEC_SHIFT    (sizeof(int) * CHAR_BIT - >> HINT_VALUE_NSEC_BITS) >> +#define HINT_VALUE_NSEC_MAX    ((1ULL << HINT_VALUE_NSEC_BITS) - 1) >> + >>   /* >>    * HINT_CLASS_BITS should be big enough to store any mp_class value. >>    * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE. >> @@ -1630,6 +1647,25 @@ hint_uuid_raw(const char *data) >>       return hint_create(MP_CLASS_UUID, val); >>   } >> +static inline hint_t >> +hint_datetime(struct datetime *date) >> +{ >> +    /* >> +     * Use at most HINT_VALUE_SECS_BITS from datetime >> +     * seconds field as a hint value, and at MSB part >> +     * of HINT_VALUE_NSEC_BITS from nanoseconds. >> +     */ >> +    int64_t secs = date->secs; >> +    int32_t nsec = date->nsec; >> +    uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 : >> +            secs - HINT_VALUE_SECS_MIN; >> +    if (val >= HINT_VALUE_SECS_MAX) >> +        val = HINT_VALUE_SECS_MAX; >> +    val <<= HINT_VALUE_NSEC_BITS; >> +    val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX; >> +    return hint_create(MP_CLASS_DATETIME, val); >> +} >> + > > > Patch increment here small (so far) ------------------------------------ diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc index 110017853..2478498ba 100644 --- a/src/box/tuple_compare.cc +++ b/src/box/tuple_compare.cc @@ -1550,8 +1550,8 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint, */ #define HINT_VALUE_SECS_BITS 36 #define HINT_VALUE_NSEC_BITS (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS) -#define HINT_VALUE_SECS_MAX ((1LL << HINT_VALUE_SECS_BITS) - 1) -#define HINT_VALUE_SECS_MIN (-(1LL << HINT_VALUE_SECS_BITS)) +#define HINT_VALUE_SECS_MAX ((1LL << (HINT_VALUE_SECS_BITS - 1)) - 1) +#define HINT_VALUE_SECS_MIN (-(1LL << (HINT_VALUE_SECS_BITS - 1))) #define HINT_VALUE_NSEC_SHIFT (sizeof(int) * CHAR_BIT - HINT_VALUE_NSEC_BITS) #define HINT_VALUE_NSEC_MAX ((1ULL << HINT_VALUE_NSEC_BITS) - 1) ------------------------------------ But please see code moves which will be done in the next version of a patchset, so all field and indices changes will become part of a single patch. Regards, Timur