Tarantool development patches archive
 help / color / mirror / Atom feed
From: Serge Petrenko <sergepetrenko@tarantool.org>
To: vdavydov.dev@gmail.com
Cc: tarantool-patches@freelists.org,
	Serge Petrenko <sergepetrenko@tarantool.org>
Subject: [PATCH 1/5] decimal: fix decimal.round() when scale == 0
Date: Wed, 17 Jul 2019 18:33:42 +0300	[thread overview]
Message-ID: <07a505e362ff037833e6501bf62a864b86694f91.1563376957.git.sergepetrenko@tarantool.org> (raw)
In-Reply-To: <cover.1563376957.git.sergepetrenko@tarantool.org>

Fixes decimal.round() with zero scale, fixes an error with
decimal.round() when rounding leads to a number with the same
precision, for example, decimal.round(9.9, 0) -> 10.

Follow-up #692
---
 src/lib/core/decimal.c    |  8 ++++----
 test/app/decimal.result   | 18 ++++++++++++++++++
 test/app/decimal.test.lua |  6 ++++++
 3 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
index 1b4f44362..1423ae418 100644
--- a/src/lib/core/decimal.c
+++ b/src/lib/core/decimal.c
@@ -172,14 +172,14 @@ decimal_round(decimal_t *dec, int scale)
 	if (scale < 0 || scale > DECIMAL_MAX_DIGITS)
 		return NULL;
 
-	if (scale > decimal_scale(dec))
+	if (scale >= decimal_scale(dec))
 		return dec;
 
-	int ndig = decimal_precision(dec) - decimal_scale(dec) + scale;
+	int ndig = MAX(decimal_precision(dec) - decimal_scale(dec) + scale, 1);
 	decContext context = {
 		ndig, /* Precision */
-		ndig - 1, /* emax */
-		-1, /* emin */
+		ndig, /* emax */
+		scale != 0 ? -1 : 0, /* emin */
 		DECIMAL_ROUNDING, /* rounding */
 		0, /* no traps */
 		0, /* zero status */
diff --git a/test/app/decimal.result b/test/app/decimal.result
index ecb189277..53ec73edc 100644
--- a/test/app/decimal.result
+++ b/test/app/decimal.result
@@ -458,3 +458,21 @@ a ^ 2.5
  | ---
  | - error: '[string "return a ^ 2.5 "]:1: decimal operation failed'
  | ...
+
+-- check correct rounding when scale = 0
+decimal.round(decimal.new(0.9), 0)
+ | ---
+ | - '1'
+ | ...
+decimal.round(decimal.new(9.9), 0)
+ | ---
+ | - '10'
+ | ...
+decimal.round(decimal.new(99.9), 0)
+ | ---
+ | - '100'
+ | ...
+decimal.round(decimal.new(99.4), 0)
+ | ---
+ | - '99'
+ | ...
diff --git a/test/app/decimal.test.lua b/test/app/decimal.test.lua
index 4aff0f2a4..20fd4d81a 100644
--- a/test/app/decimal.test.lua
+++ b/test/app/decimal.test.lua
@@ -128,3 +128,9 @@ a = decimal.new('-13')
 a ^ 2
 -- fractional powers are allowed only for positive numbers
 a ^ 2.5
+
+-- check correct rounding when scale = 0
+decimal.round(decimal.new(0.9), 0)
+decimal.round(decimal.new(9.9), 0)
+decimal.round(decimal.new(99.9), 0)
+decimal.round(decimal.new(99.4), 0)
-- 
2.20.1 (Apple Git-117)

  reply	other threads:[~2019-07-17 15:33 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-17 15:33 [PATCH 0/5] Decimal indices Serge Petrenko
2019-07-17 15:33 ` Serge Petrenko [this message]
2019-07-24 12:10   ` [PATCH 1/5] decimal: fix decimal.round() when scale == 0 Vladimir Davydov
2019-07-24 23:02   ` [tarantool-patches] " Konstantin Osipov
2019-07-17 15:33 ` [PATCH 2/5] decimal: allow to encode/decode decimals as MsgPack Serge Petrenko
2019-07-24 16:19   ` Vladimir Davydov
2019-07-24 23:08   ` [tarantool-patches] " Konstantin Osipov
2019-07-17 15:33 ` [PATCH 3/5] lua: allow to encode and decode decimals as msgpack Serge Petrenko
2019-07-24 14:11   ` Vladimir Davydov
2019-07-24 23:10   ` [tarantool-patches] " Konstantin Osipov
2019-07-17 15:33 ` [PATCH 4/5] decimal: add conversions to (u)int64_t Serge Petrenko
2019-07-24 16:17   ` Vladimir Davydov
2019-07-17 15:33 ` [PATCH 5/5] decimal: allow to index decimals Serge Petrenko
2019-07-24 16:56   ` Vladimir Davydov
2019-07-24 23:13   ` [tarantool-patches] " Konstantin Osipov
2019-07-24 23:17   ` Konstantin Osipov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=07a505e362ff037833e6501bf62a864b86694f91.1563376957.git.sergepetrenko@tarantool.org \
    --to=sergepetrenko@tarantool.org \
    --cc=tarantool-patches@freelists.org \
    --cc=vdavydov.dev@gmail.com \
    --subject='Re: [PATCH 1/5] decimal: fix decimal.round() when scale == 0' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox