From: Serge Petrenko <sergepetrenko@tarantool.org> To: vdavydov.dev@gmail.com Cc: tarantool-patches@freelists.org, Serge Petrenko <sergepetrenko@tarantool.org> Subject: [PATCH 1/5] decimal: fix decimal.round() when scale == 0 Date: Wed, 17 Jul 2019 18:33:42 +0300 [thread overview] Message-ID: <07a505e362ff037833e6501bf62a864b86694f91.1563376957.git.sergepetrenko@tarantool.org> (raw) In-Reply-To: <cover.1563376957.git.sergepetrenko@tarantool.org> Fixes decimal.round() with zero scale, fixes an error with decimal.round() when rounding leads to a number with the same precision, for example, decimal.round(9.9, 0) -> 10. Follow-up #692 --- src/lib/core/decimal.c | 8 ++++---- test/app/decimal.result | 18 ++++++++++++++++++ test/app/decimal.test.lua | 6 ++++++ 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c index 1b4f44362..1423ae418 100644 --- a/src/lib/core/decimal.c +++ b/src/lib/core/decimal.c @@ -172,14 +172,14 @@ decimal_round(decimal_t *dec, int scale) if (scale < 0 || scale > DECIMAL_MAX_DIGITS) return NULL; - if (scale > decimal_scale(dec)) + if (scale >= decimal_scale(dec)) return dec; - int ndig = decimal_precision(dec) - decimal_scale(dec) + scale; + int ndig = MAX(decimal_precision(dec) - decimal_scale(dec) + scale, 1); decContext context = { ndig, /* Precision */ - ndig - 1, /* emax */ - -1, /* emin */ + ndig, /* emax */ + scale != 0 ? -1 : 0, /* emin */ DECIMAL_ROUNDING, /* rounding */ 0, /* no traps */ 0, /* zero status */ diff --git a/test/app/decimal.result b/test/app/decimal.result index ecb189277..53ec73edc 100644 --- a/test/app/decimal.result +++ b/test/app/decimal.result @@ -458,3 +458,21 @@ a ^ 2.5 | --- | - error: '[string "return a ^ 2.5 "]:1: decimal operation failed' | ... + +-- check correct rounding when scale = 0 +decimal.round(decimal.new(0.9), 0) + | --- + | - '1' + | ... +decimal.round(decimal.new(9.9), 0) + | --- + | - '10' + | ... +decimal.round(decimal.new(99.9), 0) + | --- + | - '100' + | ... +decimal.round(decimal.new(99.4), 0) + | --- + | - '99' + | ... diff --git a/test/app/decimal.test.lua b/test/app/decimal.test.lua index 4aff0f2a4..20fd4d81a 100644 --- a/test/app/decimal.test.lua +++ b/test/app/decimal.test.lua @@ -128,3 +128,9 @@ a = decimal.new('-13') a ^ 2 -- fractional powers are allowed only for positive numbers a ^ 2.5 + +-- check correct rounding when scale = 0 +decimal.round(decimal.new(0.9), 0) +decimal.round(decimal.new(9.9), 0) +decimal.round(decimal.new(99.9), 0) +decimal.round(decimal.new(99.4), 0) -- 2.20.1 (Apple Git-117)
next prev parent reply other threads:[~2019-07-17 15:33 UTC|newest] Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-07-17 15:33 [PATCH 0/5] Decimal indices Serge Petrenko 2019-07-17 15:33 ` Serge Petrenko [this message] 2019-07-24 12:10 ` [PATCH 1/5] decimal: fix decimal.round() when scale == 0 Vladimir Davydov 2019-07-24 23:02 ` [tarantool-patches] " Konstantin Osipov 2019-07-17 15:33 ` [PATCH 2/5] decimal: allow to encode/decode decimals as MsgPack Serge Petrenko 2019-07-24 16:19 ` Vladimir Davydov 2019-07-24 23:08 ` [tarantool-patches] " Konstantin Osipov 2019-07-17 15:33 ` [PATCH 3/5] lua: allow to encode and decode decimals as msgpack Serge Petrenko 2019-07-24 14:11 ` Vladimir Davydov 2019-07-24 23:10 ` [tarantool-patches] " Konstantin Osipov 2019-07-17 15:33 ` [PATCH 4/5] decimal: add conversions to (u)int64_t Serge Petrenko 2019-07-24 16:17 ` Vladimir Davydov 2019-07-17 15:33 ` [PATCH 5/5] decimal: allow to index decimals Serge Petrenko 2019-07-24 16:56 ` Vladimir Davydov 2019-07-24 23:13 ` [tarantool-patches] " Konstantin Osipov 2019-07-24 23:17 ` Konstantin Osipov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=07a505e362ff037833e6501bf62a864b86694f91.1563376957.git.sergepetrenko@tarantool.org \ --to=sergepetrenko@tarantool.org \ --cc=tarantool-patches@freelists.org \ --cc=vdavydov.dev@gmail.com \ --subject='Re: [PATCH 1/5] decimal: fix decimal.round() when scale == 0' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox