From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id E7F7E6EC40; Wed, 18 Aug 2021 12:03:08 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E7F7E6EC40 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1629277389; bh=ju+xnSd/eZaPXnMy4K/eoKASE69VQf/TeIbA1LN4u6g=; h=To:References:Date:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=UXV9jBcIM7bXm5AG6GeXYBkYYBB6Ukt7frTPFREvRRo05UK1SFxIGmCi6hH58/N0P OVfWzNkmq98RzvHcs49zRSzI0VHdP6NpNotELY+/FrRFlD3gW+nfel0CoibTkRIRJ+ y1L7mdTSFQkJr6kVm38HOBT+rjTjLDxnIryD240A= Received: from smtp56.i.mail.ru (smtp56.i.mail.ru [217.69.128.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 92A126EC40 for ; Wed, 18 Aug 2021 12:03:07 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 92A126EC40 Received: by smtp56.i.mail.ru with esmtpa (envelope-from ) id 1mGHTW-0002Mp-Bi; Wed, 18 Aug 2021 12:03:06 +0300 To: Safin Timur References: <906cfac8b66b70eb982b5c7fe4878da39712a305.1629071531.git.tsafin@tarantool.org> <7baf64ec-6033-8c96-c57e-eb3394ec0b07@tarantool.org> <92ef16db-d5a0-6773-97c6-24a4ba9007cc@tarantool.org> Message-ID: Date: Wed, 18 Aug 2021 12:03:06 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <92ef16db-d5a0-6773-97c6-24a4ba9007cc@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: ru X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD92087353F0EC44DD9ECFD080E047A606F6525B29142351271182A05F538085040C664784AD3B2B6D93F6DC49551D65FBF4175C54A82EF8EFCCFDB5561EF6D5892 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE71D44F6E7EB16B5A3EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637B323FE155BC226618638F802B75D45FF36EB9D2243A4F8B5A6FCA7DBDB1FC311F39EFFDF887939037866D6147AF826D88D8D7774A76FEF95435415E9D6501541117882F4460429724CE54428C33FAD305F5C1EE8F4F765FC2EE5AD8F952D28FBA471835C12D1D9774AD6D5ED66289B52BA9C0B312567BB23117882F446042972877693876707352033AC447995A7AD18618001F51B5FD3F9D2E47CDBA5A96583BA9C0B312567BB2376E601842F6C81A19E625A9149C048EE7B96B19DC4093321098B145E00970258D8FC6C240DEA7642DBF02ECDB25306B2B78CF848AE20165D0A6AB1C7CE11FEE3CF7CD7A0D5AA5F2503F1AB874ED89028C4224003CC836476EA7A3FFF5B025636E2021AF6380DFAD1A18204E546F3947CB11811A4A51E3B096D1867E19FE1407959CC434672EE6371089D37D7C0E48F6C8AA50765F790063788B3B24285A3CD0EEFF80C71ABB335746BA297DBC24807EABDAD6C7F3747799A X-B7AD71C0: AC4F5C86D027EB782CDD5689AFBDA7A213B5FB47DCBC3458834459D11680B5055BEE419D3CC6110A72B678EC42716503 X-C1DE0DAB: 0D63561A33F958A565010C83975649FF63518FCAFA2A89770617ECF42C4D97C5D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA757E10A58996CBD514410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D34351729636A02D1DED4E3FF2EA4B14A94DC15571BD587BA722B23DD7C00C9A53473622A99B20E19CE1D7E09C32AA3244C02F6C6A178CC6B11361AD53B3F7509B339C99C45E8D137E983B48618A63566E0 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojuRQ/H5n28tqUvPjIcL1gPw== X-Mailru-Sender: 583F1D7ACE8F49BD31DE23046B3A8460677EF5055628B161BBDD104054B82BE8231D5400FCEF91206BB2E709EA627F343C7DDD459B58856F0E45BC603594F5A135B915D4279FF0579437F6177E88F7363CDA0F3B3F5B9367 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH v5 5/8] box, datetime: datetime comparison for indices X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Cc: tarantool-patches@dev.tarantool.org, v.shpilevoy@tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 18.08.2021 02:43, Safin Timur пишет: > On 17.08.2021 15:16, Serge Petrenko wrote: >> >> >> 16.08.2021 02:59, Timur Safin via Tarantool-patches пишет: >>> * storage hints implemented for datetime_t values; >>> * proper comparison for indices of datetime type. >>> >>> Part of #5941 >>> Part of #5946 >> >> >> Please, add a docbot request stating that it's now possible to store >> datetime values in spaces and create indexed datetime fields. > > Will use something like that: > > @TarantoolBot document > > Title: Storage support for datetime values > > It's now possible to store datetime values in spaces and create > indexed datetime fields. > > Please refer to https://github.com/tarantool/tarantool/discussions/6244 > for more detailed description of a storage schema. Looks ok to me. > >> >> >>> --- >>>   src/box/field_def.c           | 18 ++++++++ >>>   src/box/field_def.h           |  3 ++ >>>   src/box/memtx_space.c         |  3 +- >>>   src/box/tuple_compare.cc      | 57 ++++++++++++++++++++++++++ >>>   src/box/vinyl.c               |  3 +- >>>   test/engine/datetime.result   | 77 >>> +++++++++++++++++++++++++++++++++++ >>>   test/engine/datetime.test.lua | 35 ++++++++++++++++ >>>   7 files changed, 192 insertions(+), 4 deletions(-) >>>   create mode 100644 test/engine/datetime.result >>>   create mode 100644 test/engine/datetime.test.lua >>> >>> diff --git a/src/box/field_def.c b/src/box/field_def.c >>> index 2682a42ee..97033d0bb 100644 >>> --- a/src/box/field_def.c >>> +++ b/src/box/field_def.c >>> @@ -194,3 +194,21 @@ field_type_by_name(const char *name, size_t len) >>>           return FIELD_TYPE_ANY; >>>       return field_type_MAX; >>>   } >>> + >>> +const bool field_type_index_allowed[] = >>> +    { >>> +    /* [FIELD_TYPE_ANY]      = */ false, >>> +    /* [FIELD_TYPE_UNSIGNED] = */ true, >>> +    /* [FIELD_TYPE_STRING]   = */ true, >>> +    /* [FIELD_TYPE_NUMBER]   = */ true, >>> +    /* [FIELD_TYPE_DOUBLE]   = */ true, >>> +    /* [FIELD_TYPE_INTEGER]  = */ true, >>> +    /* [FIELD_TYPE_BOOLEAN]  = */ true, >>> +    /* [FIELD_TYPE_VARBINARY]= */ true, >>> +    /* [FIELD_TYPE_SCALAR]   = */ true, >>> +    /* [FIELD_TYPE_DECIMAL]  = */ true, >>> +    /* [FIELD_TYPE_UUID]     = */ true, >>> +    /* [FIELD_TYPE_ARRAY]    = */ false, >>> +    /* [FIELD_TYPE_MAP]      = */ false, >>> +    /* [FIELD_TYPE_DATETIME] = */ true, >>> +}; >> >> >> You wouldn't need that array if you moved >> FIELD_TYPE_DATETIME above FIELD_TYPE_ARRAY >> in the previous commit. >> >> Please, do so. > > Yes, will change order and also move all field support code to this > patch (as Vova recommends). > >> >> >>> diff --git a/src/box/field_def.h b/src/box/field_def.h >>> index 120b2a93d..bd02418df 100644 >>> --- a/src/box/field_def.h >>> +++ b/src/box/field_def.h >>> @@ -120,6 +120,9 @@ extern const uint32_t field_ext_type[]; >>>   extern const struct opt_def field_def_reg[]; >>>   extern const struct field_def field_def_default; >>> +/** helper table for checking allowed indices for types */ >>> +extern const bool field_type_index_allowed[]; >>> + >>>   /** >>>    * @brief Field definition >>>    * Contains information about of one tuple field. >>> diff --git a/src/box/memtx_space.c b/src/box/memtx_space.c >>> index b71318d24..1ab16122e 100644 >>> --- a/src/box/memtx_space.c >>> +++ b/src/box/memtx_space.c >>> @@ -748,8 +748,7 @@ memtx_space_check_index_def(struct space *space, >>> struct index_def *index_def) >>>       /* Check that there are no ANY, ARRAY, MAP parts */ >>>       for (uint32_t i = 0; i < key_def->part_count; i++) { >>>           struct key_part *part = &key_def->parts[i]; >>> -        if (part->type <= FIELD_TYPE_ANY || >>> -            part->type >= FIELD_TYPE_ARRAY) { >>> +        if (!field_type_index_allowed[part->type]) { >>>               diag_set(ClientError, ER_MODIFY_INDEX, >>>                    index_def->name, space_name(space), >>>                    tt_sprintf("field type '%s' is not supported", >>> diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc >>> index 9a69f2a72..110017853 100644 >>> --- a/src/box/tuple_compare.cc >>> +++ b/src/box/tuple_compare.cc >>> @@ -538,6 +538,8 @@ tuple_compare_field_with_type(const char >>> *field_a, enum mp_type a_type, >>>                              field_b, b_type); >>>       case FIELD_TYPE_UUID: >>>           return mp_compare_uuid(field_a, field_b); >>> +    case FIELD_TYPE_DATETIME: >>> +        return mp_compare_datetime(field_a, field_b); >>>       default: >>>           unreachable(); >>>           return 0; >>> @@ -1538,6 +1540,21 @@ func_index_compare_with_key(struct tuple >>> *tuple, hint_t tuple_hint, >>>   #define HINT_VALUE_DOUBLE_MAX    (exp2(HINT_VALUE_BITS - 1) - 1) >>>   #define HINT_VALUE_DOUBLE_MIN    (-exp2(HINT_VALUE_BITS - 1)) >>> +/** >>> + * We need to squeeze 64 bits of seconds and 32 bits of nanoseconds >>> + * into 60 bits of hint value. The idea is to represent wide enough >>> + * years range, and leave the rest of bits occupied from >>> nanoseconds part: >>> + * - 36 bits is enough for time range of [208BC..4147] >>> + * - for nanoseconds there is left 24 bits, which are MSB part of >>> + *   32-bit value >>> + */ >>> +#define HINT_VALUE_SECS_BITS    36 >>> +#define HINT_VALUE_NSEC_BITS    (HINT_VALUE_BITS - >>> HINT_VALUE_SECS_BITS) >>> +#define HINT_VALUE_SECS_MAX    ((1LL << HINT_VALUE_SECS_BITS) - 1) >> >> Am I missing something? >> n bits may store values from (-2^(n-1)) to 2^(n-1)-1 >> >> should be (1LL << (HINT_VALUE_SECS_BITS -1))  - 1 ? >> >> >>> +#define HINT_VALUE_SECS_MIN    (-(1LL << HINT_VALUE_SECS_BITS)) >> >> >> >> >> should be >> >> #define HINT_VALUE_SECS_MIN    (-(1LL << (HINT_VALUE_SECS_BITS - 1))) >> >> ? >> > > Yes, my definition made sense only when used as a mask (in prior > version of a code). Thus did not take into consideration a sign bit. > You absolutely correct that if seconds are signed then we have lesser > number of bits, and your definitions of > HINT_VALUE_SECS_MAX/HINT_VALUE_SECS_MIN should be used. > >> >>> +#define HINT_VALUE_NSEC_SHIFT (sizeof(int) * CHAR_BIT - >>> HINT_VALUE_NSEC_BITS) >>> +#define HINT_VALUE_NSEC_MAX    ((1ULL << HINT_VALUE_NSEC_BITS) - 1) >>> + >>>   /* >>>    * HINT_CLASS_BITS should be big enough to store any mp_class value. >>>    * Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE. >>> @@ -1630,6 +1647,25 @@ hint_uuid_raw(const char *data) >>>       return hint_create(MP_CLASS_UUID, val); >>>   } >>> +static inline hint_t >>> +hint_datetime(struct datetime *date) >>> +{ >>> +    /* >>> +     * Use at most HINT_VALUE_SECS_BITS from datetime >>> +     * seconds field as a hint value, and at MSB part >>> +     * of HINT_VALUE_NSEC_BITS from nanoseconds. >>> +     */ >>> +    int64_t secs = date->secs; >>> +    int32_t nsec = date->nsec; >>> +    uint64_t val = secs <= HINT_VALUE_SECS_MIN ? 0 : >>> +            secs - HINT_VALUE_SECS_MIN; >>> +    if (val >= HINT_VALUE_SECS_MAX) >>> +        val = HINT_VALUE_SECS_MAX; >>> +    val <<= HINT_VALUE_NSEC_BITS; >>> +    val |= (nsec >> HINT_VALUE_NSEC_SHIFT) & HINT_VALUE_NSEC_MAX; >>> +    return hint_create(MP_CLASS_DATETIME, val); >>> +} >>> + >> >> >> > > Patch increment here small (so far) > ------------------------------------ > diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc > index 110017853..2478498ba 100644 > --- a/src/box/tuple_compare.cc > +++ b/src/box/tuple_compare.cc > @@ -1550,8 +1550,8 @@ func_index_compare_with_key(struct tuple *tuple, > hint_t tuple_hint, >   */ >  #define HINT_VALUE_SECS_BITS    36 >  #define HINT_VALUE_NSEC_BITS    (HINT_VALUE_BITS - HINT_VALUE_SECS_BITS) > -#define HINT_VALUE_SECS_MAX    ((1LL << HINT_VALUE_SECS_BITS) - 1) > -#define HINT_VALUE_SECS_MIN    (-(1LL << HINT_VALUE_SECS_BITS)) > +#define HINT_VALUE_SECS_MAX    ((1LL << (HINT_VALUE_SECS_BITS - 1)) - 1) > +#define HINT_VALUE_SECS_MIN    (-(1LL << (HINT_VALUE_SECS_BITS - 1))) >  #define HINT_VALUE_NSEC_SHIFT    (sizeof(int) * CHAR_BIT - > HINT_VALUE_NSEC_BITS) >  #define HINT_VALUE_NSEC_MAX    ((1ULL << HINT_VALUE_NSEC_BITS) - 1) > > ------------------------------------ > > But please see code moves which will be done in the next version of a > patchset, so all field and indices changes will become part of a > single patch. Sure, I'll check out the new version once it's pushed. > > Regards, > Timur -- Serge Petrenko