[tarantool-patches] Re: [PATCH v1 1/1] lua: fix strange behaviour of tonumber64

[Alexander Turenko <alexander.turenko@tarantool.org>]

On Mon, Jul 16, 2018 at 04:55:51PM +0300, Vladislav Shpilevoy wrote:
> 
> 
> On 16/07/2018 16:42, Alexander Turenko wrote:
> > > > 
> > > > > 3. Why not 'result > LLONG_MAX'? As I understand, abs(LLONG_MAX) == abs(LLONG_MIN),
> > > > > it is not? (http://www.cplusplus.com/reference/climits/)
> > > > > 
> > > > 
> > > > No, LLONG_MAX is 2^63-1, but LLONG_MIN is -2^63. We want to compare
> > > > result with 2^63. We are trying to do so in platform-independent way
> > > > (hovewer unsiged unary nimus equivalence with signed one is likely
> > > > two-complement number representation property and can be violated on
> > > > other platforms).
> > > > 
> > > > Are you think we should introduce our own constant
> > > > 9223372036854775808ULL (2^63) and avoid that complex assumptions set? It
> > > 
> > > Ultimately no. We should not invent the constants.
> > > 
> > > > would be explicitly number-representation-dependent, so maybe it is
> > > > better.
> > > 
> > > Ok. Logically we want an error on -result < INT64_MIN, right?
> > > It is the same as result > -INT64_MIN. But we can not say
> > > -INT64_MIN because abs(INT64_MIN) > INT64_MAX, yes?
> > > 
> > 
> > Yes.
> > 
> > > Then lets rephrase the comparison:
> > > 
> > >      result > -INT64_MIN
> > >             |
> > >             v
> > >    result + 1 >= -INT64_MIN
> > >             |
> > >             v
> > >      result >= -INT64_MIN - 1
> > >             |
> > >             v
> > >     result >= -(INT64_MIN + 1) <- that is the solution.
> > > 
> > > As I understand, -(INT64_MIN + 1) is exactly 2^63 - 1 and
> > > fits in int64, right?
> > 
> > 2nd step should be result - 1 >= -INT64_MIN, so not it is not the
> 
> Oh, stupid error, sorry.
> 
> > decision. Overflow is unavoidable while we are trying to operate within
> > the signed type.
> 
> No, overflow is always avoidable. As an extreme solution we have int96 type,
> that is already used for overflow checks on tuple update.
> 
> As a second one I again have tried to evolve my idea of reorganization of
> result > -INT64_MIN expression:
> 
>     result > -INT64_MIN
>            |
>            v
>  result - 1 > -INT64_MIN - 1
>            |
>            v
>  INT64_MAX == -INT64_MIN - 1
>            |
>            v
>    result - 1 > INT64_MAX
> 
> Here the result is uint64_t. So to check for overflow we use
> this predicate:
> 
> result != 0 && result - 1 > INT64_MAX.
> 
> No type casts, no overflows, explicit sizes.

Here we lean on assumption that INT64_MAX == -INT64_MIN - 1, but the
question was arisen because we trying to avoid that. At least it should
be properly commented.

I don't insist, but more like approach with explicit INT64_MIN usage.

By the way, result != 0 check is redundant, because (0ULL - 1) is
0xffffffffffffffff (unsigned value) and above than INT64_MAX.

WBR, Alexander Turenko.