From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp51.i.mail.ru (smtp51.i.mail.ru [94.100.177.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 06CD74696F5 for ; Fri, 5 Jun 2020 02:43:21 +0300 (MSK) From: Vladislav Shpilevoy Date: Fri, 5 Jun 2020 01:43:08 +0200 Message-Id: <2f9b11e05fd155605435cd4bf32ffdf10a5f91cf.1591313754.git.v.shpilevoy@tarantool.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [PATCH 11/11] sql: fix mem_apply_type double type truncation List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: tarantool-patches@dev.tarantool.org, tsafin@tarantool.org, alyapunov@tarantool.org mem_apply_type(), when tried to cast a double value to an integer, used the expressions: int64_t i = (int64_t) d; uint64_t u = (uint64_t) d; To obtain integer versions of the double value, cast them back to double, and see if they are equal. Assuming that if they are, the double can be safely cast to one of them. But this is undefined behaviour. Double can't be cast to int64_t, if it is > INT64_MAX or < INT64_MIN. And can't be cast to uint64_t, if it is < 0 or > UINT64_MAX. The patch adds explicit checks for these borders before doing the cast. Part of #4609 --- src/box/sql/vdbe.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/src/box/sql/vdbe.c b/src/box/sql/vdbe.c index 5bc106b5d..6b769805c 100644 --- a/src/box/sql/vdbe.c +++ b/src/box/sql/vdbe.c @@ -324,12 +324,18 @@ mem_apply_type(struct Mem *record, enum field_type type) return 0; if ((record->flags & MEM_Real) == MEM_Real) { double d = record->u.r; - int64_t i = (int64_t) d; - uint64_t u = (uint64_t) d; - if (i == d) - mem_set_int(record, i, i <= -1); - else if (u == d) - mem_set_u64(record, u); + if (d >= 0) { + if (double_compare_uint64(d, UINT64_MAX, + 1) > 0) + return 0; + if ((double)(uint64_t)d == d) + mem_set_u64(record, (uint64_t)d); + } else { + if (double_compare_nint64(d, INT64_MIN, 1) < 0) + return 0; + if ((double)(int64_t)d == d) + mem_set_int(record, (int64_t)d, true); + } return 0; } if ((record->flags & MEM_Str) != 0) { -- 2.21.1 (Apple Git-122.3)