[tarantool-patches] Re: [PATCH v1 1/1] lua: fix strange behaviour of tonumber64

[Alexander Turenko <alexander.turenko@tarantool.org>]

> > 
> > > 3. Why not 'result > LLONG_MAX'? As I understand, abs(LLONG_MAX) == abs(LLONG_MIN),
> > > it is not? (http://www.cplusplus.com/reference/climits/)
> > > 
> > 
> > No, LLONG_MAX is 2^63-1, but LLONG_MIN is -2^63. We want to compare
> > result with 2^63. We are trying to do so in platform-independent way
> > (hovewer unsiged unary nimus equivalence with signed one is likely
> > two-complement number representation property and can be violated on
> > other platforms).
> > 
> > Are you think we should introduce our own constant
> > 9223372036854775808ULL (2^63) and avoid that complex assumptions set? It
> 
> Ultimately no. We should not invent the constants.
> 
> > would be explicitly number-representation-dependent, so maybe it is
> > better.
> 
> Ok. Logically we want an error on -result < INT64_MIN, right?
> It is the same as result > -INT64_MIN. But we can not say
> -INT64_MIN because abs(INT64_MIN) > INT64_MAX, yes?
> 

Yes.

> Then lets rephrase the comparison:
> 
>     result > -INT64_MIN
>            |
>            v
>   result + 1 >= -INT64_MIN
>            |
>            v
>     result >= -INT64_MIN - 1
>            |
>            v
>    result >= -(INT64_MIN + 1) <- that is the solution.
> 
> As I understand, -(INT64_MIN + 1) is exactly 2^63 - 1 and
> fits in int64, right?

2nd step should be result - 1 >= -INT64_MIN, so not it is not the
decision. Overflow is unavoidable while we are trying to operate within
the signed type.

WBR, Alexander Turenko.