[tarantool-patches] Re: [PATCH v1 1/1] lua: fix strange behaviour of tonumber64

[Vladislav Shpilevoy <v.shpilevoy@tarantool.org>]

On 16/07/2018 16:42, Alexander Turenko wrote:
>>>
>>>> 3. Why not 'result > LLONG_MAX'? As I understand, abs(LLONG_MAX) == abs(LLONG_MIN),
>>>> it is not? (http://www.cplusplus.com/reference/climits/)
>>>>
>>>
>>> No, LLONG_MAX is 2^63-1, but LLONG_MIN is -2^63. We want to compare
>>> result with 2^63. We are trying to do so in platform-independent way
>>> (hovewer unsiged unary nimus equivalence with signed one is likely
>>> two-complement number representation property and can be violated on
>>> other platforms).
>>>
>>> Are you think we should introduce our own constant
>>> 9223372036854775808ULL (2^63) and avoid that complex assumptions set? It
>>
>> Ultimately no. We should not invent the constants.
>>
>>> would be explicitly number-representation-dependent, so maybe it is
>>> better.
>>
>> Ok. Logically we want an error on -result < INT64_MIN, right?
>> It is the same as result > -INT64_MIN. But we can not say
>> -INT64_MIN because abs(INT64_MIN) > INT64_MAX, yes?
>>
> 
> Yes.
> 
>> Then lets rephrase the comparison:
>>
>>      result > -INT64_MIN
>>             |
>>             v
>>    result + 1 >= -INT64_MIN
>>             |
>>             v
>>      result >= -INT64_MIN - 1
>>             |
>>             v
>>     result >= -(INT64_MIN + 1) <- that is the solution.
>>
>> As I understand, -(INT64_MIN + 1) is exactly 2^63 - 1 and
>> fits in int64, right?
> 
> 2nd step should be result - 1 >= -INT64_MIN, so not it is not the

Oh, stupid error, sorry.

> decision. Overflow is unavoidable while we are trying to operate within
> the signed type.

No, overflow is always avoidable. As an extreme solution we have int96 type,
that is already used for overflow checks on tuple update.

As a second one I again have tried to evolve my idea of reorganization of
result > -INT64_MIN expression:

     result > -INT64_MIN
            |
            v
  result - 1 > -INT64_MIN - 1
            |
            v
  INT64_MAX == -INT64_MIN - 1
            |
            v
    result - 1 > INT64_MAX

Here the result is uint64_t. So to check for overflow we use
this predicate:

result != 0 && result - 1 > INT64_MAX.

No type casts, no overflows, explicit sizes.
       

> 
> WBR, Alexander Turenko.
>