* [PATCH v2 1/8] lua: fix decimal comparison with nil
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:16 ` Konstantin Osipov
2019-08-14 11:00 ` Vladimir Davydov
2019-08-08 11:55 ` [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent Serge Petrenko
` (6 subsequent siblings)
7 siblings, 2 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
Previously decimal comparison with nil failed with following error:
`expected decimal, number or string as 2 argument`.
Fix this.
Follow-up #692
---
src/lua/decimal.c | 4 ++++
test/app/decimal.result | 26 ++++++++++++++++++++++++++
test/app/decimal.test.lua | 8 ++++++++
3 files changed, 38 insertions(+)
diff --git a/src/lua/decimal.c b/src/lua/decimal.c
index 330f4c3f8..e480ec7be 100644
--- a/src/lua/decimal.c
+++ b/src/lua/decimal.c
@@ -69,6 +69,10 @@ ldecimal_##name(struct lua_State *L) { \
static int \
ldecimal_##name(struct lua_State *L) { \
assert(lua_gettop(L) == 2); \
+ if (lua_isnil(L, 1) || lua_isnil(L, 2)) { \
+ lua_pushboolean(L, false); \
+ return 1; \
+ } \
decimal_t *lhs = lua_todecimal(L, 1); \
decimal_t *rhs = lua_todecimal(L, 2); \
lua_pushboolean(L, decimal_compare(lhs, rhs) cmp 0); \
diff --git a/test/app/decimal.result b/test/app/decimal.result
index b53f4e75d..633f3681a 100644
--- a/test/app/decimal.result
+++ b/test/app/decimal.result
@@ -223,6 +223,32 @@ b
| - '0.1'
| ...
+-- check comparsion with nil
+a == nil
+ | ---
+ | - false
+ | ...
+a ~= nil
+ | ---
+ | - true
+ | ...
+a > nil
+ | ---
+ | - false
+ | ...
+a < nil
+ | ---
+ | - false
+ | ...
+a >= nil
+ | ---
+ | - false
+ | ...
+a <= nil
+ | ---
+ | - false
+ | ...
+
decimal.sqrt(a)
| ---
| - '3.1622776601683793319988935444327185337'
diff --git a/test/app/decimal.test.lua b/test/app/decimal.test.lua
index cee56d5e7..cb14dfb56 100644
--- a/test/app/decimal.test.lua
+++ b/test/app/decimal.test.lua
@@ -62,6 +62,14 @@ a ~= b
a
b
+-- check comparsion with nil
+a == nil
+a ~= nil
+a > nil
+a < nil
+a >= nil
+a <= nil
+
decimal.sqrt(a)
decimal.ln(a)
decimal.log10(a)
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 1/8] lua: fix decimal comparison with nil
2019-08-08 11:55 ` [PATCH v2 1/8] lua: fix decimal comparison with nil Serge Petrenko
@ 2019-08-12 21:16 ` Konstantin Osipov
2019-08-14 11:00 ` Vladimir Davydov
1 sibling, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:16 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 14:56]:
> Previously decimal comparison with nil failed with following error:
> `expected decimal, number or string as 2 argument`.
> Fix this.
lgtm
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 1/8] lua: fix decimal comparison with nil
2019-08-08 11:55 ` [PATCH v2 1/8] lua: fix decimal comparison with nil Serge Petrenko
2019-08-12 21:16 ` Konstantin Osipov
@ 2019-08-14 11:00 ` Vladimir Davydov
2019-08-14 22:17 ` Konstantin Osipov
1 sibling, 1 reply; 35+ messages in thread
From: Vladimir Davydov @ 2019-08-14 11:00 UTC (permalink / raw)
To: Serge Petrenko; +Cc: tarantool-patches, kostja
On Thu, Aug 08, 2019 at 02:55:52PM +0300, Serge Petrenko wrote:
> diff --git a/test/app/decimal.result b/test/app/decimal.result
> index b53f4e75d..633f3681a 100644
> --- a/test/app/decimal.result
> +++ b/test/app/decimal.result
> @@ -223,6 +223,32 @@ b
> | - '0.1'
> | ...
>
> +-- check comparsion with nil
> +a == nil
> + | ---
> + | - false
> + | ...
> +a ~= nil
> + | ---
> + | - true
> + | ...
> +a > nil
> + | ---
> + | - false
> + | ...
> +a < nil
> + | ---
> + | - false
> + | ...
> +a >= nil
> + | ---
> + | - false
> + | ...
> +a <= nil
> + | ---
> + | - false
> + | ...
== and ~= are fine, but <=, <, >, >= look weird. Besides, they won't
work with numbers and strings. So I don't think it's a good idea to
introduce this functionality for decimals. Could you please implement
comparison with nil only for == and ~= operators?
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 1/8] lua: fix decimal comparison with nil
2019-08-14 11:00 ` Vladimir Davydov
@ 2019-08-14 22:17 ` Konstantin Osipov
0 siblings, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-14 22:17 UTC (permalink / raw)
To: Vladimir Davydov; +Cc: Serge Petrenko, tarantool-patches
* Vladimir Davydov <vdavydov.dev@gmail.com> [19/08/14 14:05]:
> == and ~= are fine, but <=, <, >, >= look weird. Besides, they won't
> work with numbers and strings. So I don't think it's a good idea to
> introduce this functionality for decimals. Could you please implement
> comparison with nil only for == and ~= operators?
I checked how other data types behave in Lua:
tarantool> "" > nil
---
- error: '[string "return "" > nil"]:1: attempt to compare nil with string'
...
tarantool> "" == nil
---
- false
...
tarantool> "" ~= nil
---
- true
...
tarantool> "" < nil
---
- error: '[string "return "" < nil"]:1: attempt to compare string with nil'
...
tarantool> 1 > nil
---
- error: '[string "return 1 > nil"]:1: attempt to compare nil with number'
...
tarantool> 1 == nil
---
- false
...
So, good point, I agree.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent.
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
2019-08-08 11:55 ` [PATCH v2 1/8] lua: fix decimal comparison with nil Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:18 ` Konstantin Osipov
2019-08-14 11:56 ` Vladimir Davydov
2019-08-08 11:55 ` [PATCH v2 3/8] lua/pickle: fix a typo Serge Petrenko
` (5 subsequent siblings)
7 siblings, 2 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
When a number having a positive exponent is encoded, the internal
decPackedFromNumber function returns a negative scale, which differs
from the scale, returned by decimal_scale(). This leads to errors in
decoding. Account for negative scale in decimal_pack() and
decimal_unpack().
Follow-up #692
---
src/lib/core/decimal.c | 10 +++++++---
test/unit/decimal.c | 15 +++++++++++++--
test/unit/decimal.result | 11 ++++++++---
3 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
index 6ef351f81..840aa5dfe 100644
--- a/src/lib/core/decimal.c
+++ b/src/lib/core/decimal.c
@@ -33,6 +33,7 @@
#include "third_party/decNumber/decContext.h"
#include "third_party/decNumber/decPacked.h"
#include "lib/core/tt_static.h"
+#include "lib/msgpuck/msgpuck.h"
#include <stddef.h>
#include <stdlib.h>
#include <float.h> /* DBL_DIG */
@@ -340,12 +341,15 @@ char *
decimal_pack(char *data, const decimal_t *dec)
{
uint32_t len = decimal_len(dec);
- *data++ = decimal_scale(dec);
+ /* reserve space for resulting scale */
+ char *svp = data++;
len--;
int32_t scale;
char *tmp = (char *)decPackedFromNumber((uint8_t *)data, len, &scale, dec);
assert(tmp == data);
- assert(scale == (int32_t)decimal_scale(dec));
+ /* scale may be negative, when exponent is > 0 */
+ assert(scale == (int32_t)decimal_scale(dec) || scale < 0);
+ mp_store_u8(svp, (int8_t)scale);
(void)tmp;
data += len;
return data;
@@ -354,7 +358,7 @@ decimal_pack(char *data, const decimal_t *dec)
decimal_t *
decimal_unpack(const char **data, uint32_t len, decimal_t *dec)
{
- int32_t scale = *((*data)++);
+ int32_t scale = (int8_t)mp_load_u8(data);
len--;
decimal_t *res = decPackedToNumber((uint8_t *)*data, len, &scale, dec);
if (res)
diff --git a/test/unit/decimal.c b/test/unit/decimal.c
index b587e1f14..b55333ed7 100644
--- a/test/unit/decimal.c
+++ b/test/unit/decimal.c
@@ -91,7 +91,7 @@ char buf[32];
static int
test_pack_unpack(void)
{
- plan(146);
+ plan(151);
test_decpack("0");
test_decpack("-0");
@@ -112,13 +112,24 @@ test_pack_unpack(void)
test_decpack("99999999999999999999999999999999999999");
test_decpack("-99999999999999999999999999999999999999");
+ /* Check correct encoding of positive exponent numbers. */
+ decimal_t dec, d1;
+ decimal_from_string(&dec, "1e10");
+ uint32_t l1 = decimal_len(&dec);
+ ok(l1 == 2, "decimal_len() is small for positive exponent decimal");
+ char *b1 = decimal_pack(buf, &dec);
+ is(b1, buf + l1, "positive exponent decimal length");
+ const char *b2 = buf;
+ is(decimal_unpack(&b2, l1, &d1), &d1, "decimal_unpack() of a positive exponent decimal");
+ is(b1, b2, "decimal_unpack uses every byte packed by decimal_pack");
+ is(decimal_compare(&dec, &d1), 0, "positive exponent number is packed/unpacked correctly");
+
/* Pack an invalid decimal. */
char *b = buf;
*b++ = 1;
*b++ = '\xab';
*b++ = '\xcd';
const char *bb = buf;
- decimal_t dec;
is(decimal_unpack(&bb, 3, &dec), NULL, "unpack malformed decimal fails");
is(bb, buf, "decode malformed decimal preserves buffer position");
diff --git a/test/unit/decimal.result b/test/unit/decimal.result
index 1c72cdfab..2dd91af49 100644
--- a/test/unit/decimal.result
+++ b/test/unit/decimal.result
@@ -277,7 +277,7 @@ ok 275 - decimal_from_string(-1)
ok 276 - decimal_log10(-1) - error on wrong operands.
ok 277 - decimal_from_string(-10)
ok 278 - decimal_sqrt(-10) - error on wrong operands.
- 1..146
+ 1..151
ok 1 - decimal_len(0)
ok 2 - decimal_len(0) == len(decimal_pack(0)
ok 3 - decimal_unpack(decimal_pack(0))
@@ -422,6 +422,11 @@ ok 278 - decimal_sqrt(-10) - error on wrong operands.
ok 142 - decimal_unpack(decimal_pack(-99999999999999999999999999999999999999)) scale
ok 143 - decimal_unpack(decimal_pack(-99999999999999999999999999999999999999)) precision
ok 144 - str(decimal_unpack(decimal_pack(-99999999999999999999999999999999999999)) == -99999999999999999999999999999999999999
- ok 145 - unpack malformed decimal fails
- ok 146 - decode malformed decimal preserves buffer position
+ ok 145 - decimal_len() is small for positive exponent decimal
+ ok 146 - positive exponent decimal length
+ ok 147 - decimal_unpack() of a positive exponent decimal
+ ok 148 - decimal_unpack uses every byte packed by decimal_pack
+ ok 149 - positive exponent number is packed/unpacked correctly
+ ok 150 - unpack malformed decimal fails
+ ok 151 - decode malformed decimal preserves buffer position
ok 279 - subtests
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent.
2019-08-08 11:55 ` [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent Serge Petrenko
@ 2019-08-12 21:18 ` Konstantin Osipov
2019-08-13 9:00 ` [tarantool-patches] " Serge Petrenko
2019-08-14 11:56 ` Vladimir Davydov
1 sibling, 1 reply; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:18 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 14:56]:
> When a number having a positive exponent is encoded, the internal
> decPackedFromNumber function returns a negative scale, which differs
> from the scale, returned by decimal_scale(). This leads to errors in
> decoding. Account for negative scale in decimal_pack() and
> decimal_unpack().
I don't understand this commit.
Why do you use mp_store_u8 + a cast to unsigned, not
mp_store_i8/mp_load_i8 without a cast?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [tarantool-patches] [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent.
2019-08-12 21:18 ` Konstantin Osipov
@ 2019-08-13 9:00 ` Serge Petrenko
2019-08-14 22:21 ` Konstantin Osipov
0 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-13 9:00 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: Vladimir Davydov, tarantool-patches
> 13 авг. 2019 г., в 0:18, Konstantin Osipov <kostja@tarantool.org> написал(а):
>
> * Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 14:56]:
>> When a number having a positive exponent is encoded, the internal
>> decPackedFromNumber function returns a negative scale, which differs
>> from the scale, returned by decimal_scale(). This leads to errors in
>> decoding. Account for negative scale in decimal_pack() and
>> decimal_unpack().
>
> I don't understand this commit.
>
> Why do you use mp_store_u8 + a cast to unsigned, not
> mp_store_i8/mp_load_i8 without a cast?
We don’t have mp_store_i8/mp_load_i8, we only have
mp_load(store)_u8/16/32/64, and mp_store(load)_u8 is used to
put(get) a single byte into(from) the buffer.
Besides, I don’t know how should an implementation of mp_store_i8
look like. Will it be a cast to unsigned and a call to mp_store_u8?
I could use mp_encode_int(), but it would require an additional
byte for scales less than -31. Why waste a byte?
Similar code is already used in
mp_encode_extl/mp_decode_extl()
(yes, it was also introduced by me, but it passed the review):
(the type of ’type’ is int8_t)
data = mp_store_u8(data, type);
…
*type = mp_load_u8(data);
>
>
> --
> Konstantin Osipov, Moscow, Russia
>
--
Serge Petrenko
sergepetrenko@tarantool.org
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [tarantool-patches] [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent.
2019-08-13 9:00 ` [tarantool-patches] " Serge Petrenko
@ 2019-08-14 22:21 ` Konstantin Osipov
0 siblings, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-14 22:21 UTC (permalink / raw)
To: Serge Petrenko; +Cc: Vladimir Davydov, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/13 12:03]:
> Similar code is already used in
> mp_encode_extl/mp_decode_extl()
> (yes, it was also introduced by me, but it passed the review):
>
> (the type of ’type’ is int8_t)
> data = mp_store_u8(data, type);
> …
> *type = mp_load_u8(data);
The problem with this approach is that you lost sign in your
storage representation. Imagine this data format is read by a code
which is different from the one you wrote, and relies on msgpack
format specifiers alone. It will honestly believe that the value
is always positive.
Basically, you should follow MsgPack guidelines for choosing
format specifiers.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent.
2019-08-08 11:55 ` [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent Serge Petrenko
2019-08-12 21:18 ` Konstantin Osipov
@ 2019-08-14 11:56 ` Vladimir Davydov
1 sibling, 0 replies; 35+ messages in thread
From: Vladimir Davydov @ 2019-08-14 11:56 UTC (permalink / raw)
To: Serge Petrenko; +Cc: tarantool-patches, kostja
On Thu, Aug 08, 2019 at 02:55:53PM +0300, Serge Petrenko wrote:
> When a number having a positive exponent is encoded, the internal
> decPackedFromNumber function returns a negative scale, which differs
> from the scale, returned by decimal_scale(). This leads to errors in
> decoding. Account for negative scale in decimal_pack() and
> decimal_unpack().
>
> Follow-up #692
> ---
> src/lib/core/decimal.c | 10 +++++++---
> test/unit/decimal.c | 15 +++++++++++++--
> test/unit/decimal.result | 11 ++++++++---
> 3 files changed, 28 insertions(+), 8 deletions(-)
>
> diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
> index 6ef351f81..840aa5dfe 100644
> --- a/src/lib/core/decimal.c
> +++ b/src/lib/core/decimal.c
> @@ -33,6 +33,7 @@
> #include "third_party/decNumber/decContext.h"
> #include "third_party/decNumber/decPacked.h"
> #include "lib/core/tt_static.h"
> +#include "lib/msgpuck/msgpuck.h"
> #include <stddef.h>
> #include <stdlib.h>
> #include <float.h> /* DBL_DIG */
> @@ -340,12 +341,15 @@ char *
> decimal_pack(char *data, const decimal_t *dec)
> {
> uint32_t len = decimal_len(dec);
> - *data++ = decimal_scale(dec);
> + /* reserve space for resulting scale */
> + char *svp = data++;
> len--;
> int32_t scale;
> char *tmp = (char *)decPackedFromNumber((uint8_t *)data, len, &scale, dec);
> assert(tmp == data);
> - assert(scale == (int32_t)decimal_scale(dec));
> + /* scale may be negative, when exponent is > 0 */
> + assert(scale == (int32_t)decimal_scale(dec) || scale < 0);
This assertion now looks kinda lopsided/incomplete and therefore
confusing: we check positive scale but not negative scale. I'd rather
remove it altogether.
> + mp_store_u8(svp, (int8_t)scale);
What is the reason to use mp_store/load helpers? AFAIU plain load/store
should work just fine (as they did before this patch).
Do I understand correctly that this particular patch must be committed
to all stable branches as well?
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 3/8] lua/pickle: fix a typo
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
2019-08-08 11:55 ` [PATCH v2 1/8] lua: fix decimal comparison with nil Serge Petrenko
2019-08-08 11:55 ` [PATCH v2 2/8] decimal: fix encoding numbers with positive exponent Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:18 ` Konstantin Osipov
2019-08-14 11:12 ` Vladimir Davydov
2019-08-08 11:55 ` [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions Serge Petrenko
` (4 subsequent siblings)
7 siblings, 2 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
---
src/lua/pickle.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/lua/pickle.c b/src/lua/pickle.c
index e47ac11b4..65208b5b3 100644
--- a/src/lua/pickle.c
+++ b/src/lua/pickle.c
@@ -109,14 +109,14 @@ lbox_pack(struct lua_State *L)
case 'I':
case 'i':
/* signed and unsigned 32-bit integers */
- if (field.type != MP_UINT && field.ival != MP_INT)
+ if (field.type != MP_UINT && field.type != MP_INT)
luaL_error(L, "pickle.pack: expected 32-bit int");
luaL_region_dup(L, buf, &field.ival, sizeof(uint32_t));
break;
case 'N':
/* signed and unsigned 32-bit big endian integers */
- if (field.type != MP_UINT && field.ival != MP_INT)
+ if (field.type != MP_UINT && field.type != MP_INT)
luaL_error(L, "pickle.pack: expected 32-bit int");
field.ival = htonl(field.ival);
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 3/8] lua/pickle: fix a typo
2019-08-08 11:55 ` [PATCH v2 3/8] lua/pickle: fix a typo Serge Petrenko
@ 2019-08-12 21:18 ` Konstantin Osipov
2019-08-14 11:12 ` Vladimir Davydov
1 sibling, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:18 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
> ---
> src/lua/pickle.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
lgtm
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 3/8] lua/pickle: fix a typo
2019-08-08 11:55 ` [PATCH v2 3/8] lua/pickle: fix a typo Serge Petrenko
2019-08-12 21:18 ` Konstantin Osipov
@ 2019-08-14 11:12 ` Vladimir Davydov
2019-08-14 11:15 ` Serge Petrenko
1 sibling, 1 reply; 35+ messages in thread
From: Vladimir Davydov @ 2019-08-14 11:12 UTC (permalink / raw)
To: Serge Petrenko; +Cc: tarantool-patches, kostja
On Thu, Aug 08, 2019 at 02:55:54PM +0300, Serge Petrenko wrote:
> ---
> src/lua/pickle.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/src/lua/pickle.c b/src/lua/pickle.c
> index e47ac11b4..65208b5b3 100644
> --- a/src/lua/pickle.c
> +++ b/src/lua/pickle.c
> @@ -109,14 +109,14 @@ lbox_pack(struct lua_State *L)
> case 'I':
> case 'i':
> /* signed and unsigned 32-bit integers */
> - if (field.type != MP_UINT && field.ival != MP_INT)
> + if (field.type != MP_UINT && field.type != MP_INT)
> luaL_error(L, "pickle.pack: expected 32-bit int");
>
> luaL_region_dup(L, buf, &field.ival, sizeof(uint32_t));
> break;
> case 'N':
> /* signed and unsigned 32-bit big endian integers */
> - if (field.type != MP_UINT && field.ival != MP_INT)
> + if (field.type != MP_UINT && field.type != MP_INT)
> luaL_error(L, "pickle.pack: expected 32-bit int");
>
> field.ival = htonl(field.ival);
I assume this is a bug and so this patch should be pushed to all
maintained branches, right?
Could you please add a test for this issue?
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 3/8] lua/pickle: fix a typo
2019-08-14 11:12 ` Vladimir Davydov
@ 2019-08-14 11:15 ` Serge Petrenko
0 siblings, 0 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-14 11:15 UTC (permalink / raw)
To: Vladimir Davydov; +Cc: tarantool-patches, Konstantin Osipov
[-- Attachment #1: Type: text/plain, Size: 1333 bytes --]
> 14 авг. 2019 г., в 14:12, Vladimir Davydov <vdavydov.dev@gmail.com> написал(а):
>
> On Thu, Aug 08, 2019 at 02:55:54PM +0300, Serge Petrenko wrote:
>> ---
>> src/lua/pickle.c | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/src/lua/pickle.c b/src/lua/pickle.c
>> index e47ac11b4..65208b5b3 100644
>> --- a/src/lua/pickle.c
>> +++ b/src/lua/pickle.c
>> @@ -109,14 +109,14 @@ lbox_pack(struct lua_State *L)
>> case 'I':
>> case 'i':
>> /* signed and unsigned 32-bit integers */
>> - if (field.type != MP_UINT && field.ival != MP_INT)
>> + if (field.type != MP_UINT && field.type != MP_INT)
>> luaL_error(L, "pickle.pack: expected 32-bit int");
>>
>> luaL_region_dup(L, buf, &field.ival, sizeof(uint32_t));
>> break;
>> case 'N':
>> /* signed and unsigned 32-bit big endian integers */
>> - if (field.type != MP_UINT && field.ival != MP_INT)
>> + if (field.type != MP_UINT && field.type != MP_INT)
>> luaL_error(L, "pickle.pack: expected 32-bit int");
>>
>> field.ival = htonl(field.ival);
>
> I assume this is a bug and so this patch should be pushed to all
> maintained branches, right?
Yep
>
> Could you please add a test for this issue?
No problem
--
Serge Petrenko
sergepetrenko@tarantool.org
[-- Attachment #2: Type: text/html, Size: 8428 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
` (2 preceding siblings ...)
2019-08-08 11:55 ` [PATCH v2 3/8] lua/pickle: fix a typo Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:23 ` Konstantin Osipov
2019-08-08 11:55 ` [PATCH v2 5/8] box: rework field_def and tuple_compare to work with mp_field_type instead of mp_type Serge Petrenko
` (3 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
We are planning to add new types, such as decimal, which will all
share a single msgpack type - MP_EXT. MP_EXT is currently treated as
uninterpretable in encoders. So, in order to distinguish such new types
introduce MP_FIELD_* instead of MP_* types for use in luaL_field
structure and msgpack encoders.
New MP_FIELD_* types will correspond to MP_* types, MP_FIELD_UNKNOWN
will serve the same purpose, as MP_EXT does now, and various MP_FIELD_*
types will correspond to various MP_EXT subtypes, such as MP_DECIMAL in
future.
Prerequisite #4333
---
src/box/lua/call.c | 16 +++---
src/box/lua/execute.c | 24 ++++----
src/box/lua/tuple.c | 12 ++--
src/lib/core/mp_user_types.h | 87 +++++++++++++++++++++++++++++
src/lua/msgpack.c | 93 ++++++++++++++++---------------
src/lua/msgpack.h | 6 +-
src/lua/pickle.c | 14 ++---
src/lua/utils.c | 66 +++++++++++-----------
src/lua/utils.h | 5 +-
third_party/lua-cjson/lua_cjson.c | 28 +++++-----
third_party/lua-yaml/lyaml.cc | 22 ++++----
11 files changed, 231 insertions(+), 142 deletions(-)
create mode 100644 src/lib/core/mp_user_types.h
diff --git a/src/box/lua/call.c b/src/box/lua/call.c
index 0ac2eb7a6..189da0d1b 100644
--- a/src/box/lua/call.c
+++ b/src/box/lua/call.c
@@ -178,11 +178,11 @@ luamp_encode_call_16(lua_State *L, struct luaL_serializer *cfg,
if (luaL_tofield(L, cfg, i, &field) < 0)
return luaT_error(L);
struct tuple *tuple;
- if (field.type == MP_EXT &&
+ if (field.type == MP_FIELD_UNKNOWN &&
(tuple = luaT_istuple(L, i)) != NULL) {
/* `return ..., box.tuple.new(...), ...` */
tuple_to_mpstream(tuple, stream);
- } else if (field.type != MP_ARRAY) {
+ } else if (field.type != MP_FIELD_ARRAY) {
/*
* `return ..., scalar, ... =>
* ..., { scalar }, ...`
@@ -207,11 +207,11 @@ luamp_encode_call_16(lua_State *L, struct luaL_serializer *cfg,
if (luaL_tofield(L, cfg, 1, &root) < 0)
return luaT_error(L);
struct tuple *tuple;
- if (root.type == MP_EXT && (tuple = luaT_istuple(L, 1)) != NULL) {
+ if (root.type == MP_FIELD_UNKNOWN && (tuple = luaT_istuple(L, 1)) != NULL) {
/* `return box.tuple()` */
tuple_to_mpstream(tuple, stream);
return 1;
- } else if (root.type != MP_ARRAY) {
+ } else if (root.type != MP_FIELD_ARRAY) {
/*
* `return scalar`
* `return map`
@@ -222,7 +222,7 @@ luamp_encode_call_16(lua_State *L, struct luaL_serializer *cfg,
return 1;
}
- assert(root.type == MP_ARRAY);
+ assert(root.type == MP_FIELD_ARRAY);
if (root.size == 0) {
/* `return {}` => `{ box.tuple() }` */
mpstream_encode_array(stream, 0);
@@ -230,15 +230,15 @@ luamp_encode_call_16(lua_State *L, struct luaL_serializer *cfg,
}
/* `return { tuple, scalar, tuple }` */
- assert(root.type == MP_ARRAY && root.size > 0);
+ assert(root.type == MP_FIELD_ARRAY && root.size > 0);
for (uint32_t t = 1; t <= root.size; t++) {
lua_rawgeti(L, 1, t);
struct luaL_field field;
if (luaL_tofield(L, cfg, -1, &field) < 0)
return luaT_error(L);
- if (field.type == MP_EXT && (tuple = luaT_istuple(L, -1))) {
+ if (field.type == MP_FIELD_UNKNOWN && (tuple = luaT_istuple(L, -1))) {
tuple_to_mpstream(tuple, stream);
- } else if (field.type != MP_ARRAY) {
+ } else if (field.type != MP_FIELD_ARRAY) {
/* The first member of root table is not tuple/array */
if (t == 1) {
/*
diff --git a/src/box/lua/execute.c b/src/box/lua/execute.c
index 76ecdd541..fa234f6d2 100644
--- a/src/box/lua/execute.c
+++ b/src/box/lua/execute.c
@@ -141,15 +141,15 @@ lua_sql_bind_decode(struct lua_State *L, struct sql_bind *bind, int idx, int i)
if (luaL_tofield(L, luaL_msgpack_default, -1, &field) < 0)
return -1;
switch (field.type) {
- case MP_UINT:
+ case MP_FIELD_UINT:
bind->u64 = field.ival;
bind->bytes = sizeof(bind->u64);
break;
- case MP_INT:
+ case MP_FIELD_INT:
bind->i64 = field.ival;
bind->bytes = sizeof(bind->i64);
break;
- case MP_STR:
+ case MP_FIELD_STR:
/*
* Data should be saved in allocated memory as it
* will be poped from Lua stack.
@@ -164,38 +164,38 @@ lua_sql_bind_decode(struct lua_State *L, struct sql_bind *bind, int idx, int i)
bind->s = buf;
bind->bytes = field.sval.len;
break;
- case MP_DOUBLE:
- case MP_FLOAT:
+ case MP_FIELD_DOUBLE:
+ case MP_FIELD_FLOAT:
bind->d = field.dval;
bind->bytes = sizeof(bind->d);
break;
- case MP_NIL:
+ case MP_FIELD_NIL:
bind->bytes = 1;
break;
- case MP_BOOL:
+ case MP_FIELD_BOOL:
/* SQLite doesn't support boolean. Use int instead. */
bind->i64 = field.bval ? 1 : 0;
bind->bytes = sizeof(bind->i64);
break;
- case MP_BIN:
+ case MP_FIELD_BIN:
bind->s = mp_decode_bin(&field.sval.data, &bind->bytes);
break;
- case MP_EXT:
+ case MP_FIELD_UNKNOWN:
diag_set(ClientError, ER_SQL_BIND_TYPE, "USERDATA",
sql_bind_name(bind));
return -1;
- case MP_ARRAY:
+ case MP_FIELD_ARRAY:
diag_set(ClientError, ER_SQL_BIND_TYPE, "ARRAY",
sql_bind_name(bind));
return -1;
- case MP_MAP:
+ case MP_FIELD_MAP:
diag_set(ClientError, ER_SQL_BIND_TYPE, "MAP",
sql_bind_name(bind));
return -1;
default:
unreachable();
}
- bind->type = field.type;
+ bind->type = field_type_to_msgpack(field.type);
lua_pop(L, lua_gettop(L) - idx);
return 0;
}
diff --git a/src/box/lua/tuple.c b/src/box/lua/tuple.c
index 183c3901d..23bc73208 100644
--- a/src/box/lua/tuple.c
+++ b/src/box/lua/tuple.c
@@ -249,11 +249,11 @@ luamp_convert_key(struct lua_State *L, struct luaL_serializer *cfg,
struct luaL_field field;
if (luaL_tofield(L, cfg, index, &field) < 0)
luaT_error(L);
- if (field.type == MP_ARRAY) {
+ if (field.type == MP_FIELD_ARRAY) {
lua_pushvalue(L, index);
luamp_encode_r(L, cfg, stream, &field, 0);
lua_pop(L, 1);
- } else if (field.type == MP_NIL) {
+ } else if (field.type == MP_FIELD_NIL) {
mpstream_encode_array(stream, 0);
} else {
mpstream_encode_array(stream, 1);
@@ -270,7 +270,7 @@ luamp_encode_tuple(struct lua_State *L, struct luaL_serializer *cfg,
struct tuple *tuple = luaT_istuple(L, index);
if (tuple != NULL) {
return tuple_to_mpstream(tuple, stream);
- } else if (luamp_encode(L, cfg, stream, index) != MP_ARRAY) {
+ } else if (luamp_encode(L, cfg, stream, index) != MP_FIELD_ARRAY) {
diag_set(ClientError, ER_TUPLE_NOT_ARRAY);
luaT_error(L);
}
@@ -286,17 +286,17 @@ tuple_to_mpstream(struct tuple *tuple, struct mpstream *stream)
}
/* A MsgPack extensions handler that supports tuples */
-static enum mp_type
+static enum mp_field_type
luamp_encode_extension_box(struct lua_State *L, int idx,
struct mpstream *stream)
{
struct tuple *tuple = luaT_istuple(L, idx);
if (tuple != NULL) {
tuple_to_mpstream(tuple, stream);
- return MP_ARRAY;
+ return MP_FIELD_ARRAY;
}
- return MP_EXT;
+ return MP_FIELD_UNKNOWN;
}
/**
diff --git a/src/lib/core/mp_user_types.h b/src/lib/core/mp_user_types.h
new file mode 100644
index 000000000..d84748b71
--- /dev/null
+++ b/src/lib/core/mp_user_types.h
@@ -0,0 +1,87 @@
+#ifndef TARANTOOL_LIB_CORE_MP_USER_TYPES_H_INCLUDED
+#define TARANTOOL_LIB_CORE_MP_USER_TYPES_H_INCLUDED
+/*
+ * Copyright 2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+enum mp_field_type {
+ MP_FIELD_UNKNOWN,
+ MP_FIELD_NIL,
+ MP_FIELD_INT,
+ MP_FIELD_UINT,
+ MP_FIELD_STR,
+ MP_FIELD_BIN,
+ MP_FIELD_ARRAY,
+ MP_FIELD_MAP,
+ MP_FIELD_BOOL,
+ MP_FIELD_FLOAT,
+ MP_FIELD_DOUBLE,
+};
+
+static inline enum mp_type
+field_type_to_msgpack(enum mp_field_type type)
+{
+ switch (type) {
+ case MP_FIELD_NIL: return MP_NIL;
+ case MP_FIELD_INT: return MP_INT;
+ case MP_FIELD_UINT: return MP_UINT;
+ case MP_FIELD_STR: return MP_STR;
+ case MP_FIELD_BIN: return MP_BIN;
+ case MP_FIELD_ARRAY: return MP_ARRAY;
+ case MP_FIELD_MAP: return MP_MAP;
+ case MP_FIELD_BOOL: return MP_BOOL;
+ case MP_FIELD_FLOAT: return MP_FLOAT;
+ case MP_FIELD_DOUBLE: return MP_DOUBLE;
+ default: return MP_EXT;
+ }
+}
+
+static inline enum mp_field_type
+msgpack_to_field_type(const char *data)
+{
+ switch(mp_typeof(*data))
+ {
+ case MP_NIL: return MP_FIELD_NIL;
+ case MP_UINT: return MP_FIELD_UINT;
+ case MP_INT: return MP_FIELD_INT;
+ case MP_STR: return MP_FIELD_STR;
+ case MP_BIN: return MP_FIELD_BIN;
+ case MP_ARRAY: return MP_FIELD_ARRAY;
+ case MP_MAP: return MP_FIELD_MAP;
+ case MP_BOOL: return MP_FIELD_BOOL;
+ case MP_FLOAT: return MP_FIELD_FLOAT;
+ case MP_DOUBLE: return MP_FIELD_DOUBLE;
+ case MP_EXT:
+ default:
+ return MP_FIELD_UNKNOWN;
+ }
+}
+
+#endif
diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
index 2126988eb..6fc2b8278 100644
--- a/src/lua/msgpack.c
+++ b/src/lua/msgpack.c
@@ -52,7 +52,7 @@ luamp_error(void *error_ctx)
struct luaL_serializer *luaL_msgpack_default = NULL;
-static enum mp_type
+static enum mp_field_type
luamp_encode_extension_default(struct lua_State *L, int idx,
struct mpstream *stream);
@@ -64,14 +64,14 @@ static luamp_encode_extension_f luamp_encode_extension =
static luamp_decode_extension_f luamp_decode_extension =
luamp_decode_extension_default;
-static enum mp_type
+static enum mp_field_type
luamp_encode_extension_default(struct lua_State *L, int idx,
struct mpstream *stream)
{
(void) L;
(void) idx;
(void) stream;
- return MP_EXT;
+ return MP_FIELD_UNKNOWN;
}
void
@@ -102,45 +102,45 @@ luamp_set_decode_extension(luamp_decode_extension_f handler)
}
}
-enum mp_type
+enum mp_field_type
luamp_encode_r(struct lua_State *L, struct luaL_serializer *cfg,
struct mpstream *stream, struct luaL_field *field,
int level)
{
int top = lua_gettop(L);
- enum mp_type type;
+ enum mp_field_type type;
-restart: /* used by MP_EXT */
+restart: /* used by MP_FIELD_UNKNOWN */
switch (field->type) {
- case MP_UINT:
+ case MP_FIELD_UINT:
mpstream_encode_uint(stream, field->ival);
- return MP_UINT;
- case MP_STR:
+ return MP_FIELD_UINT;
+ case MP_FIELD_STR:
mpstream_encode_strn(stream, field->sval.data, field->sval.len);
- return MP_STR;
- case MP_BIN:
+ return MP_FIELD_STR;
+ case MP_FIELD_BIN:
mpstream_encode_strn(stream, field->sval.data, field->sval.len);
- return MP_BIN;
- case MP_INT:
+ return MP_FIELD_BIN;
+ case MP_FIELD_INT:
mpstream_encode_int(stream, field->ival);
- return MP_INT;
- case MP_FLOAT:
+ return MP_FIELD_INT;
+ case MP_FIELD_FLOAT:
mpstream_encode_float(stream, field->fval);
- return MP_FLOAT;
- case MP_DOUBLE:
+ return MP_FIELD_FLOAT;
+ case MP_FIELD_DOUBLE:
mpstream_encode_double(stream, field->dval);
- return MP_DOUBLE;
- case MP_BOOL:
+ return MP_FIELD_DOUBLE;
+ case MP_FIELD_BOOL:
mpstream_encode_bool(stream, field->bval);
- return MP_BOOL;
- case MP_NIL:
+ return MP_FIELD_BOOL;
+ case MP_FIELD_NIL:
mpstream_encode_nil(stream);
- return MP_NIL;
- case MP_MAP:
+ return MP_FIELD_NIL;
+ case MP_FIELD_MAP:
/* Map */
if (level >= cfg->encode_max_depth) {
mpstream_encode_nil(stream); /* Limit nested maps */
- return MP_NIL;
+ return MP_FIELD_NIL;
}
mpstream_encode_map(stream, field->size);
lua_pushnil(L); /* first key */
@@ -156,12 +156,12 @@ restart: /* used by MP_EXT */
lua_pop(L, 1); /* pop value */
}
assert(lua_gettop(L) == top);
- return MP_MAP;
- case MP_ARRAY:
+ return MP_FIELD_MAP;
+ case MP_FIELD_ARRAY:
/* Array */
if (level >= cfg->encode_max_depth) {
mpstream_encode_nil(stream); /* Limit nested arrays */
- return MP_NIL;
+ return MP_FIELD_NIL;
}
uint32_t size = field->size;
mpstream_encode_array(stream, size);
@@ -173,23 +173,23 @@ restart: /* used by MP_EXT */
lua_pop(L, 1);
}
assert(lua_gettop(L) == top);
- return MP_ARRAY;
- case MP_EXT:
+ return MP_FIELD_ARRAY;
+ case MP_FIELD_UNKNOWN:
/* Run trigger if type can't be encoded */
type = luamp_encode_extension(L, top, stream);
- if (type != MP_EXT)
+ if (type != MP_FIELD_UNKNOWN)
return type; /* Value has been packed by the trigger */
/* Try to convert value to serializable type */
luaL_convertfield(L, cfg, top, field);
/* handled by luaL_convertfield */
- assert(field->type != MP_EXT);
+ assert(field->type != MP_FIELD_UNKNOWN);
assert(lua_gettop(L) == top);
goto restart;
}
- return MP_EXT;
+ return MP_FIELD_UNKNOWN;
}
-enum mp_type
+enum mp_field_type
luamp_encode(struct lua_State *L, struct luaL_serializer *cfg,
struct mpstream *stream, int index)
{
@@ -205,7 +205,7 @@ luamp_encode(struct lua_State *L, struct luaL_serializer *cfg,
struct luaL_field field;
if (luaL_tofield(L, cfg, lua_gettop(L), &field) < 0)
return luaT_error(L);
- enum mp_type top_type = luamp_encode_r(L, cfg, stream, &field, 0);
+ enum mp_field_type top_type = luamp_encode_r(L, cfg, stream, &field, 0);
if (!on_top) {
lua_remove(L, top + 1); /* remove a value copy */
@@ -219,45 +219,46 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
const char **data)
{
double d;
- switch (mp_typeof(**data)) {
- case MP_UINT:
+ enum mp_field_type type = msgpack_to_field_type(*data);
+ switch (type) {
+ case MP_FIELD_UINT:
luaL_pushuint64(L, mp_decode_uint(data));
break;
- case MP_INT:
+ case MP_FIELD_INT:
luaL_pushint64(L, mp_decode_int(data));
break;
- case MP_FLOAT:
+ case MP_FIELD_FLOAT:
d = mp_decode_float(data);
luaL_checkfinite(L, cfg, d);
lua_pushnumber(L, d);
return;
- case MP_DOUBLE:
+ case MP_FIELD_DOUBLE:
d = mp_decode_double(data);
luaL_checkfinite(L, cfg, d);
lua_pushnumber(L, d);
return;
- case MP_STR:
+ case MP_FIELD_STR:
{
uint32_t len = 0;
const char *str = mp_decode_str(data, &len);
lua_pushlstring(L, str, len);
return;
}
- case MP_BIN:
+ case MP_FIELD_BIN:
{
uint32_t len = 0;
const char *str = mp_decode_bin(data, &len);
lua_pushlstring(L, str, len);
return;
}
- case MP_BOOL:
+ case MP_FIELD_BOOL:
lua_pushboolean(L, mp_decode_bool(data));
return;
- case MP_NIL:
+ case MP_FIELD_NIL:
mp_decode_nil(data);
luaL_pushnull(L);
return;
- case MP_ARRAY:
+ case MP_FIELD_ARRAY:
{
uint32_t size = mp_decode_array(data);
lua_createtable(L, size, 0);
@@ -269,7 +270,7 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
luaL_setarrayhint(L, -1);
return;
}
- case MP_MAP:
+ case MP_FIELD_MAP:
{
uint32_t size = mp_decode_map(data);
lua_createtable(L, 0, size);
@@ -282,7 +283,7 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
luaL_setmaphint(L, -1);
return;
}
- case MP_EXT:
+ case MP_FIELD_UNKNOWN:
luamp_decode_extension(L, data);
break;
}
diff --git a/src/lua/msgpack.h b/src/lua/msgpack.h
index d0ade303f..08b8d5baf 100644
--- a/src/lua/msgpack.h
+++ b/src/lua/msgpack.h
@@ -61,12 +61,12 @@ luamp_error(void *);
enum { LUAMP_ALLOC_FACTOR = 256 };
/* low-level function needed for execute_lua_call() */
-enum mp_type
+enum mp_field_type
luamp_encode_r(struct lua_State *L, struct luaL_serializer *cfg,
struct mpstream *stream, struct luaL_field *field,
int level);
-enum mp_type
+enum mp_field_type
luamp_encode(struct lua_State *L, struct luaL_serializer *cfg,
struct mpstream *stream, int index);
@@ -74,7 +74,7 @@ void
luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
const char **data);
-typedef enum mp_type
+typedef enum mp_field_type
(*luamp_encode_extension_f)(struct lua_State *, int, struct mpstream *);
/**
diff --git a/src/lua/pickle.c b/src/lua/pickle.c
index 65208b5b3..046fbba2f 100644
--- a/src/lua/pickle.c
+++ b/src/lua/pickle.c
@@ -85,7 +85,7 @@ lbox_pack(struct lua_State *L)
case 'B':
case 'b':
/* signed and unsigned 8-bit integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 8-bit int");
luaL_region_dup(L, buf, &field.ival, sizeof(uint8_t));
@@ -93,14 +93,14 @@ lbox_pack(struct lua_State *L)
case 'S':
case 's':
/* signed and unsigned 16-bit integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 16-bit int");
luaL_region_dup(L, buf, &field.ival, sizeof(uint16_t));
break;
case 'n':
/* signed and unsigned 16-bit big endian integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 16-bit int");
field.ival = (uint16_t) htons((uint16_t) field.ival);
@@ -109,14 +109,14 @@ lbox_pack(struct lua_State *L)
case 'I':
case 'i':
/* signed and unsigned 32-bit integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 32-bit int");
luaL_region_dup(L, buf, &field.ival, sizeof(uint32_t));
break;
case 'N':
/* signed and unsigned 32-bit big endian integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 32-bit int");
field.ival = htonl(field.ival);
@@ -125,7 +125,7 @@ lbox_pack(struct lua_State *L)
case 'L':
case 'l':
/* signed and unsigned 64-bit integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 64-bit int");
luaL_region_dup(L, buf, &field.ival, sizeof(uint64_t));
@@ -133,7 +133,7 @@ lbox_pack(struct lua_State *L)
case 'Q':
case 'q':
/* signed and unsigned 64-bit integers */
- if (field.type != MP_UINT && field.type != MP_INT)
+ if (field.type != MP_FIELD_UINT && field.type != MP_FIELD_INT)
luaL_error(L, "pickle.pack: expected 64-bit int");
field.ival = bswap_u64(field.ival);
diff --git a/src/lua/utils.c b/src/lua/utils.c
index 0a4bcf517..9b2f4fd6c 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -501,13 +501,13 @@ lua_field_try_serialize(struct lua_State *L)
const char *type = lua_tostring(L, -1);
if (strcmp(type, "array") == 0 || strcmp(type, "seq") == 0 ||
strcmp(type, "sequence") == 0) {
- field->type = MP_ARRAY; /* Override type */
+ field->type = MP_FIELD_ARRAY; /* Override type */
field->size = luaL_arrlen(L, 1);
/* YAML: use flow mode if __serialize == 'seq' */
if (cfg->has_compact && type[3] == '\0')
field->compact = true;
} else if (strcmp(type, "map") == 0 || strcmp(type, "mapping") == 0) {
- field->type = MP_MAP; /* Override type */
+ field->type = MP_FIELD_MAP; /* Override type */
field->size = luaL_maplen(L, 1);
/* YAML: use flow mode if __serialize == 'map' */
if (cfg->has_compact && type[3] == '\0')
@@ -562,7 +562,7 @@ lua_field_inspect_table(struct lua_State *L, struct luaL_serializer *cfg,
lua_pop(L, 1);
}
- field->type = MP_ARRAY;
+ field->type = MP_FIELD_ARRAY;
/* Calculate size and check that table can represent an array */
lua_pushnil(L);
@@ -578,7 +578,7 @@ lua_field_inspect_table(struct lua_State *L, struct luaL_serializer *cfg,
size++;
lua_pop(L, 1); /* pop the value */
}
- field->type = MP_MAP;
+ field->type = MP_FIELD_MAP;
field->size = size;
return 0;
}
@@ -594,12 +594,12 @@ lua_field_inspect_table(struct lua_State *L, struct luaL_serializer *cfg,
diag_set(LuajitError, "excessively sparse array");
return -1;
}
- field->type = MP_MAP;
+ field->type = MP_FIELD_MAP;
field->size = size;
return 0;
}
- assert(field->type == MP_ARRAY);
+ assert(field->type == MP_FIELD_ARRAY);
field->size = max;
return 0;
}
@@ -635,23 +635,23 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index,
diag_set(LuajitError, "number must not be NaN or Inf"); \
return -1; \
} \
- field->type = MP_NIL; \
+ field->type = MP_FIELD_NIL; \
}})
switch (lua_type(L, index)) {
case LUA_TNUMBER:
num = lua_tonumber(L, index);
if (isfinite(num) && modf(num, &intpart) != 0.0) {
- field->type = MP_DOUBLE;
+ field->type = MP_FIELD_DOUBLE;
field->dval = num;
} else if (num >= 0 && num < exp2(64)) {
- field->type = MP_UINT;
+ field->type = MP_FIELD_UINT;
field->ival = (uint64_t) num;
} else if (num > -exp2(63) && num < exp2(63)) {
- field->type = MP_INT;
+ field->type = MP_FIELD_INT;
field->ival = (int64_t) num;
} else {
- field->type = MP_DOUBLE;
+ field->type = MP_FIELD_DOUBLE;
field->dval = num;
CHECK_NUMBER(num);
}
@@ -664,79 +664,79 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index,
int64_t ival;
switch (cd->ctypeid) {
case CTID_BOOL:
- field->type = MP_BOOL;
+ field->type = MP_FIELD_BOOL;
field->bval = *(bool*) cdata;
return 0;
case CTID_CCHAR:
case CTID_INT8:
ival = *(int8_t *) cdata;
- field->type = (ival >= 0) ? MP_UINT : MP_INT;
+ field->type = (ival >= 0) ? MP_FIELD_UINT : MP_FIELD_INT;
field->ival = ival;
return 0;
case CTID_INT16:
ival = *(int16_t *) cdata;
- field->type = (ival >= 0) ? MP_UINT : MP_INT;
+ field->type = (ival >= 0) ? MP_FIELD_UINT : MP_FIELD_INT;
field->ival = ival;
return 0;
case CTID_INT32:
ival = *(int32_t *) cdata;
- field->type = (ival >= 0) ? MP_UINT : MP_INT;
+ field->type = (ival >= 0) ? MP_FIELD_UINT : MP_FIELD_INT;
field->ival = ival;
return 0;
case CTID_INT64:
ival = *(int64_t *) cdata;
- field->type = (ival >= 0) ? MP_UINT : MP_INT;
+ field->type = (ival >= 0) ? MP_FIELD_UINT : MP_FIELD_INT;
field->ival = ival;
return 0;
case CTID_UINT8:
- field->type = MP_UINT;
+ field->type = MP_FIELD_UINT;
field->ival = *(uint8_t *) cdata;
return 0;
case CTID_UINT16:
- field->type = MP_UINT;
+ field->type = MP_FIELD_UINT;
field->ival = *(uint16_t *) cdata;
return 0;
case CTID_UINT32:
- field->type = MP_UINT;
+ field->type = MP_FIELD_UINT;
field->ival = *(uint32_t *) cdata;
return 0;
case CTID_UINT64:
- field->type = MP_UINT;
+ field->type = MP_FIELD_UINT;
field->ival = *(uint64_t *) cdata;
return 0;
case CTID_FLOAT:
- field->type = MP_FLOAT;
+ field->type = MP_FIELD_FLOAT;
field->fval = *(float *) cdata;
CHECK_NUMBER(field->fval);
return 0;
case CTID_DOUBLE:
- field->type = MP_DOUBLE;
+ field->type = MP_FIELD_DOUBLE;
field->dval = *(double *) cdata;
CHECK_NUMBER(field->dval);
return 0;
case CTID_P_CVOID:
case CTID_P_VOID:
if (*(void **) cdata == NULL) {
- field->type = MP_NIL;
+ field->type = MP_FIELD_NIL;
return 0;
}
/* Fall through */
default:
- field->type = MP_EXT;
+ field->type = MP_FIELD_UNKNOWN;
}
return 0;
}
case LUA_TBOOLEAN:
- field->type = MP_BOOL;
+ field->type = MP_FIELD_BOOL;
field->bval = lua_toboolean(L, index);
return 0;
case LUA_TNIL:
- field->type = MP_NIL;
+ field->type = MP_FIELD_NIL;
return 0;
case LUA_TSTRING:
field->sval.data = lua_tolstring(L, index, &size);
field->sval.len = (uint32_t) size;
- field->type = MP_STR;
+ field->type = MP_FIELD_STR;
return 0;
case LUA_TTABLE:
{
@@ -748,12 +748,12 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index,
field->sval.data = NULL;
field->sval.len = 0;
if (lua_touserdata(L, index) == NULL) {
- field->type = MP_NIL;
+ field->type = MP_FIELD_NIL;
return 0;
}
/* Fall through */
default:
- field->type = MP_EXT;
+ field->type = MP_FIELD_UNKNOWN;
}
#undef CHECK_NUMBER
return 0;
@@ -765,7 +765,7 @@ luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, int idx,
{
if (idx < 0)
idx = lua_gettop(L) + idx + 1;
- assert(field->type == MP_EXT); /* must be called after tofield() */
+ assert(field->type == MP_FIELD_UNKNOWN); /* must be called after tofield() */
if (cfg->encode_load_metatables) {
int type = lua_type(L, idx);
@@ -782,14 +782,14 @@ luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, int idx,
}
}
- if (field->type == MP_EXT && cfg->encode_use_tostring)
+ if (field->type == MP_FIELD_UNKNOWN && cfg->encode_use_tostring)
lua_field_tostring(L, cfg, idx, field);
- if (field->type != MP_EXT)
+ if (field->type != MP_FIELD_UNKNOWN)
return;
if (cfg->encode_invalid_as_nil) {
- field->type = MP_NIL;
+ field->type = MP_FIELD_NIL;
return;
}
diff --git a/src/lua/utils.h b/src/lua/utils.h
index 7e7cdc0c6..4a314ab32 100644
--- a/src/lua/utils.h
+++ b/src/lua/utils.h
@@ -54,6 +54,7 @@ extern "C" {
#include <lj_meta.h>
#include "lua/error.h"
+#include "lib/core/mp_user_types.h"
struct lua_State;
struct ibuf;
@@ -287,7 +288,7 @@ struct luaL_field {
/* Array or map. */
uint32_t size;
};
- enum mp_type type;
+ enum mp_field_type type;
bool compact; /* a flag used by YAML serializer */
};
@@ -373,7 +374,7 @@ luaL_checkfield(struct lua_State *L, struct luaL_serializer *cfg, int idx,
{
if (luaL_tofield(L, cfg, idx, field) < 0)
luaT_error(L);
- if (field->type != MP_EXT)
+ if (field->type != MP_FIELD_UNKNOWN)
return;
luaL_convertfield(L, cfg, idx, field);
}
diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c
index 631c53e3f..d8244054c 100644
--- a/third_party/lua-cjson/lua_cjson.c
+++ b/third_party/lua-cjson/lua_cjson.c
@@ -348,15 +348,15 @@ static void json_append_object(lua_State *l, struct luaL_serializer *cfg,
struct luaL_field field;
luaL_checkfield(l, cfg, -2, &field);
- if (field.type == MP_UINT) {
+ if (field.type == MP_FIELD_UINT) {
strbuf_append_char(json, '"');
json_append_uint(cfg, json, field.ival);
strbuf_append_mem(json, "\":", 2);
- } else if (field.type == MP_INT) {
+ } else if (field.type == MP_FIELD_INT) {
strbuf_append_char(json, '"');
json_append_int(cfg, json, field.ival);
strbuf_append_mem(json, "\":", 2);
- } else if (field.type == MP_STR) {
+ } else if (field.type == MP_FIELD_STR) {
json_append_string(cfg, json, field.sval.data, field.sval.len);
strbuf_append_char(json, ':');
} else {
@@ -379,38 +379,38 @@ static void json_append_data(lua_State *l, struct luaL_serializer *cfg,
struct luaL_field field;
luaL_checkfield(l, cfg, -1, &field);
switch (field.type) {
- case MP_UINT:
+ case MP_FIELD_UINT:
return json_append_uint(cfg, json, field.ival);
- case MP_STR:
- case MP_BIN:
+ case MP_FIELD_STR:
+ case MP_FIELD_BIN:
return json_append_string(cfg, json, field.sval.data, field.sval.len);
- case MP_INT:
+ case MP_FIELD_INT:
return json_append_int(cfg, json, field.ival);
- case MP_FLOAT:
+ case MP_FIELD_FLOAT:
return json_append_number(cfg, json, field.fval);
- case MP_DOUBLE:
+ case MP_FIELD_DOUBLE:
return json_append_number(cfg, json, field.dval);
- case MP_BOOL:
+ case MP_FIELD_BOOL:
if (field.bval)
strbuf_append_mem(json, "true", 4);
else
strbuf_append_mem(json, "false", 5);
break;
- case MP_NIL:
+ case MP_FIELD_NIL:
json_append_nil(cfg, json);
break;
- case MP_MAP:
+ case MP_FIELD_MAP:
if (current_depth >= cfg->encode_max_depth)
return json_append_nil(cfg, json); /* Limit nested maps */
json_append_object(l, cfg, current_depth, json);
return;
- case MP_ARRAY:
+ case MP_FIELD_ARRAY:
/* Array */
if (current_depth >= cfg->encode_max_depth)
return json_append_nil(cfg, json); /* Limit nested arrays */
json_append_array(l, cfg, current_depth, json, field.size);
return;
- case MP_EXT:
+ case MP_FIELD_UNKNOWN:
/* handled by luaL_convertfield */
assert(false);
return;
diff --git a/third_party/lua-yaml/lyaml.cc b/third_party/lua-yaml/lyaml.cc
index 46c98bde1..23d10335b 100644
--- a/third_party/lua-yaml/lyaml.cc
+++ b/third_party/lua-yaml/lyaml.cc
@@ -620,33 +620,33 @@ static int dump_node(struct lua_yaml_dumper *dumper)
int top = lua_gettop(dumper->L);
luaL_checkfield(dumper->L, dumper->cfg, top, &field);
switch(field.type) {
- case MP_UINT:
+ case MP_FIELD_UINT:
snprintf(buf, sizeof(buf) - 1, "%" PRIu64, field.ival);
buf[sizeof(buf) - 1] = 0;
str = buf;
len = strlen(buf);
break;
- case MP_INT:
+ case MP_FIELD_INT:
snprintf(buf, sizeof(buf) - 1, "%" PRIi64, field.ival);
buf[sizeof(buf) - 1] = 0;
str = buf;
len = strlen(buf);
break;
- case MP_FLOAT:
+ case MP_FIELD_FLOAT:
fpconv_g_fmt(buf, field.fval, dumper->cfg->encode_number_precision);
str = buf;
len = strlen(buf);
break;
- case MP_DOUBLE:
+ case MP_FIELD_DOUBLE:
fpconv_g_fmt(buf, field.dval, dumper->cfg->encode_number_precision);
str = buf;
len = strlen(buf);
break;
- case MP_ARRAY:
+ case MP_FIELD_ARRAY:
return dump_array(dumper, &field);
- case MP_MAP:
+ case MP_FIELD_MAP:
return dump_table(dumper, &field);
- case MP_STR:
+ case MP_FIELD_STR:
str = lua_tolstring(dumper->L, -1, &len);
if (yaml_is_null(str, len) || yaml_is_bool(str, len, &unused) ||
lua_isnumber(dumper->L, -1)) {
@@ -672,13 +672,13 @@ static int dump_node(struct lua_yaml_dumper *dumper)
break;
}
/* Fall through */
- case MP_BIN:
+ case MP_FIELD_BIN:
is_binary = 1;
tobase64(dumper->L, -1);
str = lua_tolstring(dumper->L, -1, &len);
tag = (yaml_char_t *) LUAYAML_TAG_PREFIX "binary";
break;
- case MP_BOOL:
+ case MP_FIELD_BOOL:
if (field.bval) {
str = "true";
len = 4;
@@ -687,12 +687,12 @@ static int dump_node(struct lua_yaml_dumper *dumper)
len = 5;
}
break;
- case MP_NIL:
+ case MP_FIELD_NIL:
style = YAML_PLAIN_SCALAR_STYLE;
str = "null";
len = 4;
break;
- case MP_EXT:
+ case MP_FIELD_UNKNOWN:
assert(0); /* checked by luaL_checkfield() */
break;
}
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions
2019-08-08 11:55 ` [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions Serge Petrenko
@ 2019-08-12 21:23 ` Konstantin Osipov
2019-08-13 13:15 ` [tarantool-patches] " Serge Petrenko
0 siblings, 1 reply; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:23 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
> We are planning to add new types, such as decimal, which will all
> share a single msgpack type - MP_EXT. MP_EXT is currently treated as
> uninterpretable in encoders. So, in order to distinguish such new types
> introduce MP_FIELD_* instead of MP_* types for use in luaL_field
> structure and msgpack encoders.
> New MP_FIELD_* types will correspond to MP_* types, MP_FIELD_UNKNOWN
> will serve the same purpose, as MP_EXT does now, and various MP_FIELD_*
> types will correspond to various MP_EXT subtypes, such as MP_DECIMAL in
> future.
uhg, so there is enum field_type, there is enum mp_type, and these
are not enough, let's have a yet another enum...
Sounds like you should be able to get by with enum field_type and
enum mp_type, no?
Look at mp_type when you can, and when you get MP_EXT or is
otherwise confused, look at field_type?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [tarantool-patches] [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions
2019-08-12 21:23 ` Konstantin Osipov
@ 2019-08-13 13:15 ` Serge Petrenko
2019-08-14 22:23 ` Konstantin Osipov
0 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-13 13:15 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: Vladimir Davydov, tarantool-patches
> 13 авг. 2019 г., в 0:23, Konstantin Osipov <kostja@tarantool.org> написал(а):
>
> * Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
>
>> We are planning to add new types, such as decimal, which will all
>> share a single msgpack type - MP_EXT. MP_EXT is currently treated as
>> uninterpretable in encoders. So, in order to distinguish such new types
>> introduce MP_FIELD_* instead of MP_* types for use in luaL_field
>> structure and msgpack encoders.
>> New MP_FIELD_* types will correspond to MP_* types, MP_FIELD_UNKNOWN
>> will serve the same purpose, as MP_EXT does now, and various MP_FIELD_*
>> types will correspond to various MP_EXT subtypes, such as MP_DECIMAL in
>> future.
>
> uhg, so there is enum field_type, there is enum mp_type, and these
> are not enough, let's have a yet another enum...
>
> Sounds like you should be able to get by with enum field_type and
> enum mp_type, no?
>
Sorry for the confusion. I guess mp_field_type is just a bad name.
It has nothing to do with enum field_type. It just represents all the
messagepack types, including the extension types.
It helps to eliminate a switch for extension type every time we get an
MP_EXT. So, it only deals with mp_type and mp_user_type.
> Look at mp_type when you can, and when you get MP_EXT or is
> otherwise confused, look at field_type?
That’s what I’ve done in previous revision, but Vladimir didn’t like it.
And having implemented both variants, I agree that this enum simplifies
the code quite a lot. This is especially noticeable with patch 6.
So, my proposal is to change the name to something better.
value_type? mp_value_type maybe?
>
> --
> Konstantin Osipov, Moscow, Russia
>
--
Serge Petrenko
sergepetrenko@tarantool.org
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [tarantool-patches] [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions
2019-08-13 13:15 ` [tarantool-patches] " Serge Petrenko
@ 2019-08-14 22:23 ` Konstantin Osipov
2019-08-15 8:27 ` Serge Petrenko
0 siblings, 1 reply; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-14 22:23 UTC (permalink / raw)
To: Serge Petrenko; +Cc: Vladimir Davydov, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/13 16:44]:
> Sorry for the confusion. I guess mp_field_type is just a bad name.
> It has nothing to do with enum field_type. It just represents all the
> messagepack types, including the extension types.
No it doesn't. Please read here what messagepack types are:
https://en.wikipedia.org/wiki/MessagePack
It stores messagepack format codes and extension types. This is
confusing.
> It helps to eliminate a switch for extension type every time we get an
> MP_EXT. So, it only deals with mp_type and mp_user_type.
I get that.
> > Look at mp_type when you can, and when you get MP_EXT or is
> > otherwise confused, look at field_type?
>
> That’s what I’ve done in previous revision, but Vladimir didn’t like it.
> And having implemented both variants, I agree that this enum simplifies
> the code quite a lot. This is especially noticeable with patch 6.
>
> So, my proposal is to change the name to something better.
> value_type? mp_value_type maybe?
Could you please show me the previous version of the patch?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [tarantool-patches] [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions
2019-08-14 22:23 ` Konstantin Osipov
@ 2019-08-15 8:27 ` Serge Petrenko
2019-08-16 8:06 ` Konstantin Osipov
0 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-15 8:27 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: Vladimir Davydov, tarantool-patches
[-- Attachment #1: Type: text/plain, Size: 46918 bytes --]
> 15 авг. 2019 г., в 1:23, Konstantin Osipov <kostja@tarantool.org> написал(а):
>
> * Serge Petrenko <sergepetrenko@tarantool.org> [19/08/13 16:44]:
>> Sorry for the confusion. I guess mp_field_type is just a bad name.
>> It has nothing to do with enum field_type. It just represents all the
>> messagepack types, including the extension types.
>
> No it doesn't. Please read here what messagepack types are:
> https://en.wikipedia.org/wiki/MessagePack
>
> It stores messagepack format codes and extension types. This is
> confusing.
>
>> It helps to eliminate a switch for extension type every time we get an
>> MP_EXT. So, it only deals with mp_type and mp_user_type.
>
> I get that.
>
>>> Look at mp_type when you can, and when you get MP_EXT or is
>>> otherwise confused, look at field_type?
>>
>> That’s what I’ve done in previous revision, but Vladimir didn’t like it.
>> And having implemented both variants, I agree that this enum simplifies
>> the code quite a lot. This is especially noticeable with patch 6.
>>
>> So, my proposal is to change the name to something better.
>> value_type? mp_value_type maybe?
>
> Could you please show me the previous version of the patch?
Here’s the old version of the patch on encoding/decoding decimals in lua.
This patch is based on top of the one introducing mp_encode/decode_decimal,
they’re not squashed yet in this revision.
=================================================
It is now possible to insert decimals into spaces, but only into
unindexed fields.
Part-of #4333
---
extra/exports | 4 ++
src/lib/core/decimal.c | 10 +++-
src/lib/core/mp_user_types.h | 3 +-
src/lib/core/mpstream.c | 11 ++++
src/lib/core/mpstream.h | 4 ++
src/lua/decimal.c | 10 ++--
src/lua/decimal.h | 5 ++
src/lua/msgpack.c | 52 +++++++++++++----
src/lua/msgpackffi.lua | 33 +++++++++++
src/lua/utils.c | 16 ++++-
src/lua/utils.h | 8 ++-
test/app/decimal.result | 106 +++++++++++++++++-----------------
test/app/msgpack.result | 41 +++++++++++++
test/app/msgpack.test.lua | 15 +++++
third_party/lua-yaml/lyaml.cc <http://lyaml.cc/> | 6 ++
15 files changed, 248 insertions(+), 76 deletions(-)
diff --git a/extra/exports b/extra/exports
index b8c42c0df..7b84a1452 100644
--- a/extra/exports
+++ b/extra/exports
@@ -62,8 +62,12 @@ PMurHash32_Result
crc32_calc
mp_encode_double
mp_encode_float
+mp_encode_decimal
mp_decode_double
mp_decode_float
+mp_decode_extl
+mp_sizeof_decimal
+decimal_unpack
log_type
say_set_log_level
diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
index 1423ae418..2c69a773c 100644
--- a/src/lib/core/decimal.c
+++ b/src/lib/core/decimal.c
@@ -33,6 +33,7 @@
#include "third_party/decNumber/decContext.h"
#include "third_party/decNumber/decPacked.h"
#include "lib/core/tt_static.h"
+#include "lib/msgpuck/msgpuck.h"
#include <stddef.h>
#include <stdlib.h>
#include <float.h> /* DBL_DIG */
@@ -312,12 +313,15 @@ char *
decimal_pack(char *data, const decimal_t *dec)
{
uint32_t len = decimal_len(dec);
- *data++ = decimal_scale(dec);
+ /* reserve space for resulting scale */
+ char *svp = data++;
len--;
int32_t scale;
char *tmp = (char *)decPackedFromNumber((uint8_t *)data, len, &scale, dec);
assert(tmp == data);
- assert(scale == (int32_t)decimal_scale(dec));
+ /* scale may be negative, when exponent is > 0 */
+ assert(scale == (int32_t)decimal_scale(dec) || scale < 0);
+ mp_store_u8(svp, (int8_t)scale);
(void)tmp;
data += len;
return data;
@@ -326,7 +330,7 @@ decimal_pack(char *data, const decimal_t *dec)
decimal_t *
decimal_unpack(const char **data, uint32_t len, decimal_t *dec)
{
- int32_t scale = *((*data)++);
+ int32_t scale = (int8_t)mp_load_u8(data);
len--;
decimal_t *res = decPackedToNumber((uint8_t *)*data, len, &scale, dec);
if (res)
diff --git a/src/lib/core/mp_user_types.h b/src/lib/core/mp_user_types.h
index 9158b40d3..8211e3e79 100644
--- a/src/lib/core/mp_user_types.h
+++ b/src/lib/core/mp_user_types.h
@@ -32,7 +32,8 @@
*/
enum mp_user_type {
- MP_DECIMAL = 0
+ MP_UNKNOWN = 0,
+ MP_DECIMAL = 1
};
#endif
diff --git a/src/lib/core/mpstream.c b/src/lib/core/mpstream.c
index 8b7276ab1..a46b7962b 100644
--- a/src/lib/core/mpstream.c
+++ b/src/lib/core/mpstream.c
@@ -33,6 +33,7 @@
#include <assert.h>
#include <stdint.h>
#include "msgpuck.h"
+#include "mp_decimal.h"
void
mpstream_reserve_slow(struct mpstream *stream, size_t size)
@@ -186,6 +187,16 @@ mpstream_encode_binl(struct mpstream *stream, uint32_t len)
mpstream_advance(stream, pos - data);
}
+void
+mpstream_encode_decimal(struct mpstream *stream, decimal_t *val)
+{
+ char *data = mpstream_reserve(stream, mp_sizeof_decimal(val));
+ if (data == NULL)
+ return;
+ char *pos = mp_encode_decimal(data, val);
+ mpstream_advance(stream, pos - data);
+}
+
void
mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n)
{
diff --git a/src/lib/core/mpstream.h b/src/lib/core/mpstream.h
index 69fa76f7f..44af28cb5 100644
--- a/src/lib/core/mpstream.h
+++ b/src/lib/core/mpstream.h
@@ -32,6 +32,7 @@
*/
#include "diag.h"
+#include "decimal.h"
#if defined(__cplusplus)
extern "C" {
@@ -136,6 +137,9 @@ mpstream_encode_bool(struct mpstream *stream, bool val);
void
mpstream_encode_binl(struct mpstream *stream, uint32_t len);
+void
+mpstream_encode_decimal(struct mpstream *stream, decimal_t *val);
+
/** Copies n bytes from memory area src to stream. */
void
mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n);
diff --git a/src/lua/decimal.c b/src/lua/decimal.c
index e548cdb9d..ab8d85f75 100644
--- a/src/lua/decimal.c
+++ b/src/lua/decimal.c
@@ -31,7 +31,7 @@
#include "lua/decimal.h"
#include "lib/core/decimal.h"
-#include "lua/utils.h"
+#include "lua/utils.h" /* CTID_DECIMAL, ... */
#include <lua.h>
#include <lauxlib.h>
@@ -69,16 +69,18 @@ ldecimal_##name(struct lua_State *L) { \
static int \
ldecimal_##name(struct lua_State *L) { \
assert(lua_gettop(L) == 2); \
+ if (lua_isnil(L, 1) || lua_isnil(L, 2)) { \
+ lua_pushboolean(L, false); \
+ return 1; \
+ } \
decimal_t *lhs = lua_todecimal(L, 1); \
decimal_t *rhs = lua_todecimal(L, 2); \
lua_pushboolean(L, decimal_compare(lhs, rhs) cmp 0); \
return 1; \
}
-static uint32_t CTID_DECIMAL;
-
/** Push a new decimal on the stack and return a pointer to it. */
-static decimal_t *
+decimal_t *
lua_pushdecimal(struct lua_State *L)
{
decimal_t *res = luaL_pushcdata(L, CTID_DECIMAL);
diff --git a/src/lua/decimal.h b/src/lua/decimal.h
index 0485d11ef..b5c0e54b4 100644
--- a/src/lua/decimal.h
+++ b/src/lua/decimal.h
@@ -31,12 +31,17 @@
#ifndef TARANTOOL_LUA_DECIMAL_H_INCLUDED
#define TARANTOOL_LUA_DECIMAL_H_INCLUDED
+#include "lib/core/decimal.h"
+
#if defined(__cplusplus)
extern "C" {
#endif /* defined(__cplusplus) */
struct lua_State;
+decimal_t *
+lua_pushdecimal(struct lua_State *L);
+
void
tarantool_lua_decimal_init(struct lua_State *L);
diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
index 2126988eb..3cb7d7dcd 100644
--- a/src/lua/msgpack.c
+++ b/src/lua/msgpack.c
@@ -41,6 +41,10 @@
#include <small/region.h>
#include <small/ibuf.h>
+#include "lua/decimal.h"
+#include "lib/core/decimal.h"
+#include "lib/core/mp_user_types.h"
+
#include <fiber.h>
void
@@ -175,16 +179,23 @@ restart: /* used by MP_EXT */
assert(lua_gettop(L) == top);
return MP_ARRAY;
case MP_EXT:
- /* Run trigger if type can't be encoded */
- type = luamp_encode_extension(L, top, stream);
- if (type != MP_EXT)
- return type; /* Value has been packed by the trigger */
- /* Try to convert value to serializable type */
- luaL_convertfield(L, cfg, top, field);
- /* handled by luaL_convertfield */
- assert(field->type != MP_EXT);
- assert(lua_gettop(L) == top);
- goto restart;
+ switch (field->ext_type)
+ {
+ case MP_DECIMAL:
+ mpstream_encode_decimal(stream, field->decval);
+ return MP_EXT;
+ case MP_UNKNOWN:
+ /* Run trigger if type can't be encoded */
+ type = luamp_encode_extension(L, top, stream);
+ if (type != MP_EXT)
+ return type; /* Value has been packed by the trigger */
+ /* Try to convert value to serializable type */
+ luaL_convertfield(L, cfg, top, field);
+ /* handled by luaL_convertfield */
+ assert(field->type != MP_EXT || field->ext_type != MP_UNKNOWN);
+ assert(lua_gettop(L) == top);
+ goto restart;
+ }
}
return MP_EXT;
}
@@ -283,9 +294,28 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
return;
}
case MP_EXT:
- luamp_decode_extension(L, data);
+ {
+ uint32_t len;
+ int8_t type;
+ len = mp_decode_extl(data, &type);
+ switch (type) {
+ case MP_DECIMAL:
+ {
+ decimal_t *dec = lua_pushdecimal(L);
+ dec = decimal_unpack(data, len, dec);
+ if (dec == NULL) {
+ lua_pop(L, -1);
+ luaL_error(L, "msgpack.decode: "
+ "invalid MsgPack.");
+ }
+ return;
+ }
+ default:
+ luamp_decode_extension(L, data);
+ }
break;
}
+ }
}
diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua
index bfeedbc4b..4c799eed0 100644
--- a/src/lua/msgpackffi.lua
+++ b/src/lua/msgpackffi.lua
@@ -17,10 +17,18 @@ char *
mp_encode_float(char *data, float num);
char *
mp_encode_double(char *data, double num);
+char *
+mp_encode_decimal(char *data, decimal_t *dec);
+uint32_t
+mp_sizeof_decimal(const decimal_t *dec);
float
mp_decode_float(const char **data);
double
mp_decode_double(const char **data);
+uint32_t
+mp_decode_extl(const char **data, int8_t *type);
+decimal_t *
+decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
]])
local strict_alignment = (jit.arch == 'arm')
@@ -117,6 +125,11 @@ local function encode_double(buf, num)
builtin.mp_encode_double(p, num)
end
+local function encode_decimal(buf, num)
+ local p = buf:alloc(builtin.mp_sizeof_decimal(num))
+ builtin.mp_encode_decimal(p, num)
+end
+
local function encode_int(buf, num)
if num >= 0 then
if num <= 0x7f then
@@ -294,6 +307,7 @@ on_encode(ffi.typeof('const unsigned char'), encode_int)
on_encode(ffi.typeof('bool'), encode_bool_cdata)
on_encode(ffi.typeof('float'), encode_float)
on_encode(ffi.typeof('double'), encode_double)
+on_encode(ffi.typeof('decimal_t'), encode_decimal)
--------------------------------------------------------------------------------
-- Decoder
@@ -473,6 +487,23 @@ local function decode_map(data, size)
return setmetatable(map, msgpack.map_mt)
end
+local function decode_ext(data)
+ local t = ffi.new("int8_t[1]")
+ -- mp_decode_extl and mp_decode_decimal
+ -- need type code
+ data[0] = data[0] - 1
+ local old_data = data[0]
+ local len = builtin.mp_decode_extl(data, t)
+ --MP_DECIMAL
+ if t[0] == 1 then
+ local num = ffi.new("decimal_t")
+ builtin.decimal_unpack(data, len, num)
+ return num
+ else
+ error("Unsupported extension type")
+ end
+end
+
local decoder_hint = {
--[[{{{ MP_BIN]]
[0xc4] = function(data) return decode_str(data, decode_u8(data)) end;
@@ -528,6 +559,8 @@ decode_r = function(data)
return false
elseif c == 0xc3 then
return true
+ elseif c >= 0xd4 and c <= 0xd8 or c >= 0xc7 and c <= 0xc9 then
+ return decode_ext(data)
else
local fun = decoder_hint[c];
assert (type(fun) == "function")
diff --git a/src/lua/utils.c b/src/lua/utils.c
index 0a4bcf517..47cf030ab 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -45,6 +45,8 @@ static uint32_t CTID_STRUCT_IBUF;
static uint32_t CTID_STRUCT_IBUF_PTR;
uint32_t CTID_CHAR_PTR;
uint32_t CTID_CONST_CHAR_PTR;
+uint32_t CTID_DECIMAL;
+
void *
luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
@@ -723,6 +725,12 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index,
/* Fall through */
default:
field->type = MP_EXT;
+ if (cd->ctypeid == CTID_DECIMAL) {
+ field->ext_type = MP_DECIMAL;
+ field->decval = (decimal_t *) cdata;
+ } else {
+ field->ext_type = MP_UNKNOWN;
+ }
}
return 0;
}
@@ -754,6 +762,7 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index,
/* Fall through */
default:
field->type = MP_EXT;
+ field->ext_type = MP_UNKNOWN;
}
#undef CHECK_NUMBER
return 0;
@@ -765,7 +774,7 @@ luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, int idx,
{
if (idx < 0)
idx = lua_gettop(L) + idx + 1;
- assert(field->type == MP_EXT); /* must be called after tofield() */
+ assert(field->type == MP_EXT && field->ext_type == MP_UNKNOWN); /* must be called after tofield() */
if (cfg->encode_load_metatables) {
int type = lua_type(L, idx);
@@ -782,10 +791,11 @@ luaL_convertfield(struct lua_State *L, struct luaL_serializer *cfg, int idx,
}
}
- if (field->type == MP_EXT && cfg->encode_use_tostring)
+ if (field->type == MP_EXT && field->ext_type == MP_UNKNOWN &&
+ cfg->encode_use_tostring)
lua_field_tostring(L, cfg, idx, field);
- if (field->type != MP_EXT)
+ if (field->type != MP_EXT || field->ext_type != MP_UNKNOWN)
return;
if (cfg->encode_invalid_as_nil) {
diff --git a/src/lua/utils.h b/src/lua/utils.h
index 7e7cdc0c6..3ca43292b 100644
--- a/src/lua/utils.h
+++ b/src/lua/utils.h
@@ -54,6 +54,8 @@ extern "C" {
#include <lj_meta.h>
#include "lua/error.h"
+#include "lib/core/mp_user_types.h"
+#include "lib/core/decimal.h" /* decimal_t */
struct lua_State;
struct ibuf;
@@ -69,6 +71,8 @@ extern struct ibuf *tarantool_lua_ibuf;
extern uint32_t CTID_CONST_CHAR_PTR;
extern uint32_t CTID_CHAR_PTR;
+extern uint32_t CTID_DECIMAL;
+
/** \cond public */
@@ -286,8 +290,10 @@ struct luaL_field {
bool bval;
/* Array or map. */
uint32_t size;
+ decimal_t *decval;
};
enum mp_type type;
+ enum mp_user_type ext_type;
bool compact; /* a flag used by YAML serializer */
};
@@ -373,7 +379,7 @@ luaL_checkfield(struct lua_State *L, struct luaL_serializer *cfg, int idx,
{
if (luaL_tofield(L, cfg, idx, field) < 0)
luaT_error(L);
- if (field->type != MP_EXT)
+ if (field->type != MP_EXT || field->ext_type != MP_UNKNOWN)
return;
luaL_convertfield(L, cfg, idx, field);
}
diff --git a/test/app/decimal.result b/test/app/decimal.result
index 53ec73edc..334c9fbab 100644
--- a/test/app/decimal.result
+++ b/test/app/decimal.result
@@ -12,77 +12,77 @@ ffi = require('ffi')
-- check various constructors
decimal.new('1234.5678')
| ---
- | - '1234.5678'
+ | - 1234.5678
| ...
decimal.new('1e6')
| ---
- | - '1000000'
+ | - 1000000
| ...
decimal.new('-6.234612e2')
| ---
- | - '-623.4612'
+ | - -623.4612
| ...
-- check (u)int16/32/64_t
decimal.new(2ULL ^ 63)
| ---
- | - '9223372036854775808'
+ | - 9223372036854775808
| ...
decimal.new(123456789123456789ULL)
| ---
- | - '123456789123456789'
+ | - 123456789123456789
| ...
decimal.new(-123456789123456789LL)
| ---
- | - '-123456789123456789'
+ | - -123456789123456789
| ...
decimal.new(ffi.new('uint8_t', 231))
| ---
- | - '231'
+ | - 231
| ...
decimal.new(ffi.new('int8_t', -113))
| ---
- | - '-113'
+ | - -113
| ...
decimal.new(ffi.new('uint16_t', 65535))
| ---
- | - '65535'
+ | - 65535
| ...
decimal.new(ffi.new('int16_t', -31263))
| ---
- | - '-31263'
+ | - -31263
| ...
decimal.new(ffi.new('uint32_t', 4123123123))
| ---
- | - '4123123123'
+ | - 4123123123
| ...
decimal.new(ffi.new('int32_t', -2123123123))
| ---
- | - '-2123123123'
+ | - -2123123123
| ...
decimal.new(ffi.new('float', 128.5))
| ---
- | - '128.5'
+ | - 128.5
| ...
decimal.new(ffi.new('double', 128.5))
| ---
- | - '128.5'
+ | - 128.5
| ...
decimal.new(1)
| ---
- | - '1'
+ | - 1
| ...
decimal.new(-1)
| ---
- | - '-1'
+ | - -1
| ...
decimal.new(2^64)
| ---
- | - '18446744073709600000'
+ | - 18446744073709600000
| ...
decimal.new(2^(-20))
| ---
- | - '0.00000095367431640625'
+ | - 0.00000095367431640625
| ...
-- incorrect constructions
@@ -128,38 +128,38 @@ a = decimal.new('10')
| ...
a
| ---
- | - '10'
+ | - 10
| ...
b = decimal.new('0.1')
| ---
| ...
b
| ---
- | - '0.1'
+ | - 0.1
| ...
a + b
| ---
- | - '10.1'
+ | - 10.1
| ...
a - b
| ---
- | - '9.9'
+ | - 9.9
| ...
a * b
| ---
- | - '1.0'
+ | - 1.0
| ...
a / b
| ---
- | - '100'
+ | - 100
| ...
a ^ b
| ---
- | - '1.2589254117941672104239541063958006061'
+ | - 1.2589254117941672104239541063958006061
| ...
b ^ a
| ---
- | - '0.0000000001'
+ | - 0.0000000001
| ...
-a + -b == -(a + b)
| ---
@@ -167,11 +167,11 @@ b ^ a
| ...
a
| ---
- | - '10'
+ | - 10
| ...
b
| ---
- | - '0.1'
+ | - 0.1
| ...
a < b
@@ -216,28 +216,28 @@ a ~= b
| ...
a
| ---
- | - '10'
+ | - 10
| ...
b
| ---
- | - '0.1'
+ | - 0.1
| ...
decimal.sqrt(a)
| ---
- | - '3.1622776601683793319988935444327185337'
+ | - 3.1622776601683793319988935444327185337
| ...
decimal.ln(a)
| ---
- | - '2.3025850929940456840179914546843642076'
+ | - 2.3025850929940456840179914546843642076
| ...
decimal.log10(a)
| ---
- | - '1'
+ | - 1
| ...
decimal.exp(a)
| ---
- | - '22026.465794806716516957900645284244366'
+ | - 22026.465794806716516957900645284244366
| ...
a == decimal.ln(decimal.exp(a))
| ---
@@ -261,7 +261,7 @@ a + -a == 0
| ...
a
| ---
- | - '10'
+ | - 10
| ...
a = decimal.new('1.1234567891234567891234567891234567891')
@@ -269,7 +269,7 @@ a = decimal.new('1.1234567891234567891234567891234567891')
| ...
a
| ---
- | - '1.1234567891234567891234567891234567891'
+ | - 1.1234567891234567891234567891234567891
| ...
decimal.precision(a)
| ---
@@ -285,7 +285,7 @@ decimal.round(a, 37) == a
| ...
a
| ---
- | - '1.1234567891234567891234567891234567891'
+ | - 1.1234567891234567891234567891234567891
| ...
a = decimal.round(a, 36)
| ---
@@ -309,19 +309,19 @@ decimal.round(a, -5) == a
| ...
decimal.round(a, 7)
| ---
- | - '1.1234568'
+ | - 1.1234568
| ...
decimal.round(a, 3)
| ---
- | - '1.123'
+ | - 1.123
| ...
decimal.round(a, 0)
| ---
- | - '1'
+ | - 1
| ...
a
| ---
- | - '1.123456789123456789123456789123456789'
+ | - 1.123456789123456789123456789123456789
| ...
decimal.ln(0)
@@ -334,7 +334,7 @@ decimal.ln(-1)
| ...
decimal.ln(1)
| ---
- | - '0'
+ | - 0
| ...
decimal.log10(0)
| ---
@@ -346,7 +346,7 @@ decimal.log10(-1)
| ...
decimal.log10(1)
| ---
- | - '0'
+ | - 0
| ...
decimal.exp(88)
| ---
@@ -354,7 +354,7 @@ decimal.exp(88)
| ...
decimal.exp(87)
| ---
- | - '60760302250568721495223289381302760753'
+ | - 60760302250568721495223289381302760753
| ...
decimal.sqrt(-5)
| ---
@@ -362,7 +362,7 @@ decimal.sqrt(-5)
| ...
decimal.sqrt(5)
| ---
- | - '2.2360679774997896964091736687312762354'
+ | - 2.2360679774997896964091736687312762354
| ...
-- various incorrect operands
@@ -408,11 +408,11 @@ a ^ 2
| ...
a ^ 1.9
| ---
- | - '1258925411794167210423954106395800606.1'
+ | - 1258925411794167210423954106395800606.1
| ...
a * '1e18'
| ---
- | - '10000000000000000000000000000000000000'
+ | - 10000000000000000000000000000000000000
| ...
a = decimal.new(string.rep('9', 38))
| ---
@@ -435,7 +435,7 @@ a + '0.5'
| ...
a + '0.4'
| ---
- | - '99999999999999999999999999999999999999'
+ | - 99999999999999999999999999999999999999
| ...
a / 0.5
| ---
@@ -451,7 +451,7 @@ a = decimal.new('-13')
| ...
a ^ 2
| ---
- | - '169'
+ | - 169
| ...
-- fractional powers are allowed only for positive numbers
a ^ 2.5
@@ -462,17 +462,17 @@ a ^ 2.5
-- check correct rounding when scale = 0
decimal.round(decimal.new(0.9), 0)
| ---
- | - '1'
+ | - 1
| ...
decimal.round(decimal.new(9.9), 0)
| ---
- | - '10'
+ | - 10
| ...
decimal.round(decimal.new(99.9), 0)
| ---
- | - '100'
+ | - 100
| ...
decimal.round(decimal.new(99.4), 0)
| ---
- | - '99'
+ | - 99
| ...
diff --git a/test/app/msgpack.result b/test/app/msgpack.result
index a67c05d38..4b5aec784 100644
--- a/test/app/msgpack.result
+++ b/test/app/msgpack.result
@@ -252,3 +252,44 @@ msgpack.decode(ffi.cast('char *', '\x04\x05\x06'), -1)
---
- error: 'msgpack.decode: size can''t be negative'
...
+--
+-- gh-4333: msgpack encode/decode decimals.
+--
+decimal = require('decimal')
+---
+...
+a = decimal.new('1e37')
+---
+...
+b = decimal.new('1e-38')
+---
+...
+c = decimal.new('1')
+---
+...
+d = decimal.new('0.1234567')
+---
+...
+e = decimal.new('123.4567')
+---
+...
+msgpack.decode(msgpack.encode(a)) == a
+---
+- true
+...
+msgpack.decode(msgpack.encode(b)) == b
+---
+- true
+...
+msgpack.decode(msgpack.encode(c)) == c
+---
+- true
+...
+msgpack.decode(msgpack.encode(d)) == d
+---
+- true
+...
+msgpack.decode(msgpack.encode(e)) == e
+---
+- true
+...
diff --git a/test/app/msgpack.test.lua b/test/app/msgpack.test.lua
index e0880ac22..9224d870a 100644
--- a/test/app/msgpack.test.lua
+++ b/test/app/msgpack.test.lua
@@ -84,3 +84,18 @@ size = msgpack.encode(100, buf)
-- is not negative.
--
msgpack.decode(ffi.cast('char *', '\x04\x05\x06'), -1)
+
+--
+-- gh-4333: msgpack encode/decode decimals.
+--
+decimal = require('decimal')
+a = decimal.new('1e37')
+b = decimal.new('1e-38')
+c = decimal.new('1')
+d = decimal.new('0.1234567')
+e = decimal.new('123.4567')
+msgpack.decode(msgpack.encode(a)) == a
+msgpack.decode(msgpack.encode(b)) == b
+msgpack.decode(msgpack.encode(c)) == c
+msgpack.decode(msgpack.encode(d)) == d
+msgpack.decode(msgpack.encode(e)) == e
diff --git a/third_party/lua-yaml/lyaml.cc <http://lyaml.cc/> b/third_party/lua-yaml/lyaml.cc <http://lyaml.cc/>
index 46c98bde1..8ff2867e3 100644
--- a/third_party/lua-yaml/lyaml.cc <http://lyaml.cc/>
+++ b/third_party/lua-yaml/lyaml.cc <http://lyaml.cc/>
@@ -49,6 +49,7 @@ extern "C" {
#include "b64.h"
} /* extern "C" */
#include "lua/utils.h"
+#include "lib/core/decimal.h"
#define LUAYAML_TAG_PREFIX "tag:yaml.org <http://yaml.org/>,2002:"
@@ -693,6 +694,11 @@ static int dump_node(struct lua_yaml_dumper *dumper)
len = 4;
break;
case MP_EXT:
+ if (field.ext_type == MP_DECIMAL) {
+ str = decimal_to_string(field.decval);
+ len = strlen(str);
+ break;
+ }
assert(0); /* checked by luaL_checkfield() */
break;
}
--
2.20.1 (Apple Git-117)
=======================================
Here’s the old version of the patch adding decimal indices
=======================================
Indices can now be built over decimal fields.
A new field type - 'decimal' is introduced.
Decimal values may be stored in 'decimal' columns, as well as in
'scalar' and 'any' columns.
Closes #4333
@TarantoolBot document
Title: Document decimal field type.
Decimals may now be stored in spaces. A corresponding field type is
introduced: 'decimal'. Decimal values are also allowed in 'scalar' and
'number' fields.
'decimal' field type is appropriate for both memtx HASH and TREE
indices, as well as for vinyl TREE index.
---
src/box/field_def.c | 43 +++++--
src/box/field_def.h | 16 ++-
src/box/key_def.h | 2 +-
src/box/tuple_compare.cc <http://tuple_compare.cc/> | 160 ++++++++++++++++++++++++-
src/box/tuple_format.c | 2 +-
src/lib/core/decimal.h | 8 ++
src/lib/core/mp_decimal.h | 8 ++
test/engine/decimal.result | 226 +++++++++++++++++++++++++++++++++++
test/engine/decimal.test.lua | 65 ++++++++++
9 files changed, 510 insertions(+), 20 deletions(-)
create mode 100644 test/engine/decimal.result
create mode 100644 test/engine/decimal.test.lua
diff --git a/src/box/field_def.c b/src/box/field_def.c
index 346042b98..da06e6bde 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -52,17 +52,32 @@ const uint32_t field_mp_type[] = {
/* [FIELD_TYPE_UNSIGNED] = */ 1U << MP_UINT,
/* [FIELD_TYPE_STRING] = */ 1U << MP_STR,
/* [FIELD_TYPE_NUMBER] = */ (1U << MP_UINT) | (1U << MP_INT) |
- (1U << MP_FLOAT) | (1U << MP_DOUBLE),
+ (1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_EXT),
/* [FIELD_TYPE_INTEGER] = */ (1U << MP_UINT) | (1U << MP_INT),
/* [FIELD_TYPE_BOOLEAN] = */ 1U << MP_BOOL,
/* [FIELD_TYPE_VARBINARY] = */ 1U << MP_BIN,
/* [FIELD_TYPE_SCALAR] = */ (1U << MP_UINT) | (1U << MP_INT) |
(1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_STR) |
- (1U << MP_BIN) | (1U << MP_BOOL),
+ (1U << MP_BIN) | (1U << MP_BOOL) | (1U << MP_EXT),
+ /* [FIELD_TYPE_DECIMAL] = */ 1U << MP_EXT,
/* [FIELD_TYPE_ARRAY] = */ 1U << MP_ARRAY,
/* [FIELD_TYPE_MAP] = */ (1U << MP_MAP),
};
+const uint32_t field_ext_type[] = {
+ /* [FIELD_TYPE_ANY] = */ UINT32_MAX & ~(1U << MP_UNKNOWN),
+ /* [FIELD_TYPE_UNSIGNED] = */ 0,
+ /* [FIELD_TYPE_STRING] = */ 0,
+ /* [FIELD_TYPE_NUMBER] = */ 1U << MP_DECIMAL,
+ /* [FIELD_TYPE_INTEGER] = */ 0,
+ /* [FIELD_TYPE_BOOLEAN] = */ 0,
+ /* [FIELD_TYPE_VARBINARY] = */ 0,
+ /* [FIELD_TYPE_SCALAR] = */ 1U << MP_DECIMAL,
+ /* [FIELD_TYPE_DECIMAL] = */ 1U << MP_DECIMAL,
+ /* [FIELD_TYPE_ARRAY] = */ 0,
+ /* [FIELD_TYPE_MAP] = */ 0,
+};
+
const char *field_type_strs[] = {
/* [FIELD_TYPE_ANY] = */ "any",
/* [FIELD_TYPE_UNSIGNED] = */ "unsigned",
@@ -72,6 +87,7 @@ const char *field_type_strs[] = {
/* [FIELD_TYPE_BOOLEAN] = */ "boolean",
/* [FIELD_TYPE_VARBINARY] = */"varbinary",
/* [FIELD_TYPE_SCALAR] = */ "scalar",
+ /* [FIELD_TYPE_DECIMAL] = */ "decimal",
/* [FIELD_TYPE_ARRAY] = */ "array",
/* [FIELD_TYPE_MAP] = */ "map",
};
@@ -98,17 +114,18 @@ field_type_by_name_wrapper(const char *str, uint32_t len)
* values can be stored in the j type.
*/
static const bool field_type_compatibility[] = {
- /* ANY UNSIGNED STRING NUMBER INTEGER BOOLEAN VARBINARY SCALAR ARRAY MAP */
-/* ANY */ true, false, false, false, false, false, false, false, false, false,
-/* UNSIGNED */ true, true, false, true, true, false, false, true, false, false,
-/* STRING */ true, false, true, false, false, false, false, true, false, false,
-/* NUMBER */ true, false, false, true, false, false, false, true, false, false,
-/* INTEGER */ true, false, false, true, true, false, false, true, false, false,
-/* BOOLEAN */ true, false, false, false, false, true, false, true, false, false,
-/* VARBINARY*/ true, false, false, false, false, false, true, true, false, false,
-/* SCALAR */ true, false, false, false, false, false, false, true, false, false,
-/* ARRAY */ true, false, false, false, false, false, false, false, true, false,
-/* MAP */ true, false, false, false, false, false, false, false, false, true,
+ /* ANY UNSIGNED STRING NUMBER INTEGER BOOLEAN VARBINARY SCALAR DECIMAL ARRAY MAP */
+/* ANY */ true, false, false, false, false, false, false, false, false, false, false,
+/* UNSIGNED */ true, true, false, true, true, false, false, true, false, false, false,
+/* STRING */ true, false, true, false, false, false, false, true, false, false, false,
+/* NUMBER */ true, false, false, true, false, false, false, true, false, false, false,
+/* INTEGER */ true, false, false, true, true, false, false, true, false, false, false,
+/* BOOLEAN */ true, false, false, false, false, true, false, true, false, false, false,
+/* VARBINARY*/ true, false, false, false, false, false, true, true, false, false, false,
+/* SCALAR */ true, false, false, false, false, false, false, true, false, false, false,
+/* DECIMAL */ true, false, false, true, false, false, false, true, true, false, false,
+/* ARRAY */ true, false, false, false, false, false, false, false, false, true, false,
+/* MAP */ true, false, false, false, false, false, false, false, false, false, true,
};
bool
diff --git a/src/box/field_def.h b/src/box/field_def.h
index c1a7ec0a9..10dc9fc13 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -37,6 +37,7 @@
#include <limits.h>
#include <msgpuck.h>
#include "opt_def.h"
+#include "lib/core/mp_user_types.h"
#if defined(__cplusplus)
extern "C" {
@@ -58,6 +59,7 @@ enum field_type {
FIELD_TYPE_BOOLEAN,
FIELD_TYPE_VARBINARY,
FIELD_TYPE_SCALAR,
+ FIELD_TYPE_DECIMAL,
FIELD_TYPE_ARRAY,
FIELD_TYPE_MAP,
field_type_MAX
@@ -109,8 +111,9 @@ field_type_by_name(const char *name, size_t len);
/* MsgPack type names */
extern const char *mp_type_strs[];
-/** A helper table for field_mp_type_is_compatible */
+/** Two helper tables for field_mp_type_is_compatible */
extern const uint32_t field_mp_type[];
+extern const uint32_t field_ext_type[];
extern const struct opt_def field_def_reg[];
extern const struct field_def field_def_default;
@@ -142,15 +145,20 @@ struct field_def {
struct Expr *default_value_expr;
};
-/** Checks if mp_type (MsgPack) is compatible with field type. */
+/** Checks if mp_typeof(data) is compatible with field type. */
static inline bool
-field_mp_type_is_compatible(enum field_type type, enum mp_type mp_type,
+field_mp_type_is_compatible(enum field_type type, const char *data,
bool is_nullable)
{
+ enum mp_type mp_type = mp_typeof(*data);
+ int8_t ext_type = MP_UNKNOWN;
assert(type < field_type_MAX);
assert((size_t)mp_type < CHAR_BIT * sizeof(*field_mp_type));
uint32_t mask = field_mp_type[type] | (is_nullable * (1U << MP_NIL));
- return (mask & (1U << mp_type)) != 0;
+ if (mp_type != MP_EXT)
+ return (mask & (1U << mp_type)) != 0;
+ mp_decode_extl(&data, &ext_type);
+ return (field_ext_type[type] & (1U << ext_type)) != 0;
}
static inline bool
diff --git a/src/box/key_def.h b/src/box/key_def.h
index f4a1a8fd1..ed3354ade 100644
--- a/src/box/key_def.h
+++ b/src/box/key_def.h
@@ -505,7 +505,7 @@ static inline int
key_part_validate(enum field_type key_type, const char *key,
uint32_t field_no, bool is_nullable)
{
- if (unlikely(!field_mp_type_is_compatible(key_type, mp_typeof(*key),
+ if (unlikely(!field_mp_type_is_compatible(key_type, key,
is_nullable))) {
diag_set(ClientError, ER_KEY_PART_TYPE, field_no,
field_type_strs[key_type]);
diff --git a/src/box/tuple_compare.cc <http://tuple_compare.cc/> b/src/box/tuple_compare.cc <http://tuple_compare.cc/>
index 95a0f58c9..6ab28f662 100644
--- a/src/box/tuple_compare.cc <http://tuple_compare.cc/>
+++ b/src/box/tuple_compare.cc <http://tuple_compare.cc/>
@@ -33,6 +33,9 @@
#include "coll/coll.h"
#include "trivia/util.h" /* NOINLINE */
#include <math.h>
+#include "lib/core/decimal.h"
+#include "lib/core/mp_decimal.h"
+#include "lib/core/mp_user_types.h"
/* {{{ tuple_compare */
@@ -87,7 +90,7 @@ static enum mp_class mp_classes[] = {
/* .MP_BOOL = */ MP_CLASS_BOOL,
/* .MP_FLOAT = */ MP_CLASS_NUMBER,
/* .MP_DOUBLE = */ MP_CLASS_NUMBER,
- /* .MP_BIN = */ MP_CLASS_BIN
+ /* .MP_EXT = */ MP_CLASS_NUMBER /* requires additional parsing */
};
#define COMPARE_RESULT(a, b) (a < b ? -1 : a > b)
@@ -264,6 +267,50 @@ mp_compare_double_any_number(double lhs, const char *rhs,
return k * COMPARE_RESULT(lqbit, rqbit);
}
+static int
+mp_compare_decimal_any_number(decimal_t *lhs, const char *rhs,
+ enum mp_type rhs_type, int k)
+{
+ decimal_t rhs_dec;
+ switch (rhs_type) {
+ case MP_FLOAT:
+ {
+ double d = mp_decode_float(&rhs);
+ decimal_from_double(&rhs_dec, d);
+ break;
+ }
+ case MP_DOUBLE:
+ {
+ double d = mp_decode_double(&rhs);
+ decimal_from_double(&rhs_dec, d);
+ break;
+ }
+ case MP_INT:
+ {
+ int64_t num = mp_decode_int(&rhs);
+ decimal_from_int64(&rhs_dec, num);
+ break;
+ }
+ case MP_UINT:
+ {
+ uint64_t num = mp_decode_uint(&rhs);
+ decimal_from_uint64(&rhs_dec, num);
+ break;
+ }
+ case MP_EXT:
+ {
+ int8_t ext_type;
+ uint32_t len = mp_decode_extl(&rhs, &ext_type);
+ assert(ext_type == MP_DECIMAL);
+ decimal_unpack(&rhs, len, &rhs_dec);
+ break;
+ }
+ default:
+ unreachable();
+ }
+ return k * decimal_compare(lhs, &rhs_dec);
+}
+
static int
mp_compare_number_with_type(const char *lhs, enum mp_type lhs_type,
const char *rhs, enum mp_type rhs_type)
@@ -271,6 +318,21 @@ mp_compare_number_with_type(const char *lhs, enum mp_type lhs_type,
assert(mp_classof(lhs_type) == MP_CLASS_NUMBER);
assert(mp_classof(rhs_type) == MP_CLASS_NUMBER);
+ /*
+ * test decimals first, so that we don't have to
+ * account for them in other comarators.
+ */
+ decimal_t dec;
+ if (rhs_type == MP_EXT) {
+ return mp_compare_decimal_any_number(
+ mp_decode_decimal(&rhs, &dec), lhs, lhs_type, -1
+ );
+ }
+ if (lhs_type == MP_EXT) {
+ return mp_compare_decimal_any_number(
+ mp_decode_decimal(&lhs, &dec), rhs, rhs_type, 1
+ );
+ }
if (rhs_type == MP_FLOAT) {
return mp_compare_double_any_number(
mp_decode_float(&rhs), lhs, lhs_type, -1
@@ -410,6 +472,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
return coll != NULL ?
mp_compare_scalar_coll(field_a, field_b, coll) :
mp_compare_scalar(field_a, field_b);
+ case FIELD_TYPE_DECIMAL:
+ return mp_compare_number(field_a, field_b);
default:
unreachable();
return 0;
@@ -443,6 +507,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
mp_compare_scalar_coll(field_a, field_b, coll) :
mp_compare_scalar_with_type(field_a, a_type,
field_b, b_type);
+ case FIELD_TYPE_DECIMAL:
+ return mp_compare_number(field_a, field_b);
default:
unreachable();
return 0;
@@ -1356,6 +1422,25 @@ static const comparator_with_key_signature cmp_wk_arr[] = {
#define HINT_VALUE_DOUBLE_MAX (exp2(HINT_VALUE_BITS - 1) - 1)
#define HINT_VALUE_DOUBLE_MIN (-exp2(HINT_VALUE_BITS - 1))
+/**
+ * Max and min decimal numbers whose integral parts fit
+ * in a hint value.
+ */
+static const decimal_t HINT_VALUE_DECIMAL_MAX = {
+ 18, /* decimal digits */
+ 0, /* exponent */
+ 0, /* no special bits. */
+ {487, 423, 303, 752, 460, 576} /* 576,460,752,303,423,488 = 2^59 - 1 */
+ /* 59 == HINT_VALUE_BITS - 1 */
+};
+
+static const decimal_t HINT_VALUE_DECIMAL_MIN = {
+ 18, /* decimal digits */
+ 0, /* exponent */
+ 0x80, /* negative bit */
+ {488, 423, 303, 752, 460, 576} /* 576,460,752,303,423,488 = 2^59 */
+};
+
/*
* HINT_CLASS_BITS should be big enough to store any mp_class value.
* Note, ((1 << HINT_CLASS_BITS) - 1) is reserved for HINT_NONE.
@@ -1415,6 +1500,25 @@ hint_double(double d)
return hint_create(MP_CLASS_NUMBER, val);
}
+static inline hint_t
+hint_decimal(decimal_t *dec)
+{
+ uint64_t val = 0;
+ int64_t num;
+ if (decimal_compare(dec, &HINT_VALUE_DECIMAL_MAX) >= 0)
+ val = HINT_VALUE_MAX;
+ else if (decimal_compare(dec, &HINT_VALUE_DECIMAL_MIN) <= 0)
+ val = 0;
+ else {
+ dec = decimal_to_int64(dec, &num);
+ /* We've checked boundaries above. */
+ assert(dec != NULL);
+ val = num - HINT_VALUE_INT_MIN;
+ }
+
+ return hint_create(MP_CLASS_NUMBER, val);
+}
+
static inline uint64_t
hint_str_raw(const char *s, uint32_t len)
{
@@ -1491,12 +1595,42 @@ field_hint_number(const char *field)
return hint_double(mp_decode_float(&field));
case MP_DOUBLE:
return hint_double(mp_decode_double(&field));
+ case MP_EXT:
+ {
+ int8_t ext_type;
+ uint32_t len = mp_decode_extl(&field, &ext_type);
+ switch(ext_type) {
+ case MP_DECIMAL:
+ {
+ decimal_t dec;
+ decimal_t *res;
+ /*
+ * The effect of mp_decode_extl() +
+ * decimal_unpack() is the same that
+ * the one of mp_decode_decimal().
+ */
+ res = decimal_unpack(&field, len, &dec);
+ assert(res != NULL);
+ return hint_decimal(res);
+ }
+ default:
+ unreachable();
+ }
+ }
default:
unreachable();
}
return HINT_NONE;
}
+static inline hint_t
+field_hint_decimal(const char *field)
+{
+ assert(mp_typeof(*field) == MP_EXT);
+ decimal_t dec;
+ return hint_decimal(mp_decode_decimal(&field, &dec));
+}
+
static inline hint_t
field_hint_string(const char *field, struct coll *coll)
{
@@ -1536,6 +1670,25 @@ field_hint_scalar(const char *field, struct coll *coll)
case MP_BIN:
len = mp_decode_binl(&field);
return hint_bin(field, len);
+ case MP_EXT:
+ {
+ int8_t ext_type;
+ uint32_t len = mp_decode_extl(&field, &ext_type);
+ switch(ext_type) {
+ case MP_DECIMAL:
+ {
+ decimal_t dec;
+ /*
+ * The effect of mp_decode_extl() +
+ * decimal_unpack() is the same that
+ * the one of mp_decode_decimal().
+ */
+ return hint_decimal(decimal_unpack(&field, len, &dec));
+ }
+ default:
+ unreachable();
+ }
+ }
default:
unreachable();
}
@@ -1563,6 +1716,8 @@ field_hint(const char *field, struct coll *coll)
return field_hint_varbinary(field);
case FIELD_TYPE_SCALAR:
return field_hint_scalar(field, coll);
+ case FIELD_TYPE_DECIMAL:
+ return field_hint_decimal(field);
default:
unreachable();
}
@@ -1668,6 +1823,9 @@ key_def_set_hint_func(struct key_def *def)
case FIELD_TYPE_SCALAR:
key_def_set_hint_func<FIELD_TYPE_SCALAR>(def);
break;
+ case FIELD_TYPE_DECIMAL:
+ key_def_set_hint_func<FIELD_TYPE_DECIMAL>(def);
+ break;
default:
/* Invalid key definition. */
def->key_hint = NULL;
diff --git a/src/box/tuple_format.c b/src/box/tuple_format.c
index 02fadf1cf..5bb659b87 100644
--- a/src/box/tuple_format.c
+++ b/src/box/tuple_format.c
@@ -1165,7 +1165,7 @@ tuple_format_iterator_next(struct tuple_format_iterator *it,
* defined in format.
*/
bool is_nullable = tuple_field_is_nullable(field);
- if (!field_mp_type_is_compatible(field->type, mp_typeof(*entry->data),
+ if (!field_mp_type_is_compatible(field->type, entry->data,
is_nullable) != 0) {
diag_set(ClientError, ER_FIELD_TYPE,
tuple_field_path(field),
diff --git a/src/lib/core/decimal.h b/src/lib/core/decimal.h
index a4e7683c7..6a152b08c 100644
--- a/src/lib/core/decimal.h
+++ b/src/lib/core/decimal.h
@@ -37,6 +37,10 @@
#include "third_party/decNumber/decNumber.h"
#include <stdint.h>
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
typedef decNumber decimal_t;
/**
@@ -207,4 +211,8 @@ decimal_pack(char *data, const decimal_t *dec);
decimal_t *
decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
#endif /* TARANTOOL_LIB_CORE_DECIMAL_H_INCLUDED */
diff --git a/src/lib/core/mp_decimal.h b/src/lib/core/mp_decimal.h
index a991a5f16..778529068 100644
--- a/src/lib/core/mp_decimal.h
+++ b/src/lib/core/mp_decimal.h
@@ -34,6 +34,10 @@
#include "decimal.h"
#include <stdint.h>
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
/**
* \brief Calculate exact buffer size needed to store a decimal
* pointed to by \a dec.
@@ -59,4 +63,8 @@ mp_decode_decimal(const char **data, decimal_t *dec);
char *
mp_encode_decimal(char *data, const decimal_t *dec);
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
#endif
diff --git a/test/engine/decimal.result b/test/engine/decimal.result
new file mode 100644
index 000000000..16cd335e0
--- /dev/null
+++ b/test/engine/decimal.result
@@ -0,0 +1,226 @@
+-- test-run result file version 2
+env = require('test_run')
+ | ---
+ | ...
+test_run = env.new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+
+decimal = require('decimal')
+ | ---
+ | ...
+ffi = require('ffi')
+ | ---
+ | ...
+
+_ = box.schema.space.create('test', {engine=engine})
+ | ---
+ | ...
+_ = box.space.test:create_index('pk', {parts={1,'decimal'}})
+ | ---
+ | ...
+
+test_run:cmd('setopt delimiter ";"')
+ | ---
+ | - true
+ | ...
+for i = 0,16 do
+ box.space.test:insert{decimal.new((i-8)/4)}
+end;
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ""');
+ | ---
+ | - true
+ | ...
+
+box.space.test:select{}
+ | ---
+ | - - [-2]
+ | - [-1.75]
+ | - [-1.5]
+ | - [-1.25]
+ | - [-1]
+ | - [-0.75]
+ | - [-0.5]
+ | - [-0.25]
+ | - [0]
+ | - [0.25]
+ | - [0.5]
+ | - [0.75]
+ | - [1]
+ | - [1.25]
+ | - [1.5]
+ | - [1.75]
+ | - [2]
+ | ...
+
+-- check invalid values
+box.space.test:insert{1.23}
+ | ---
+ | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
+ | ...
+box.space.test:insert{'str'}
+ | ---
+ | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
+ | ...
+box.space.test:insert{ffi.new('uint64_t', 0)}
+ | ---
+ | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
+ | ...
+-- check duplicates
+box.space.test:insert{decimal.new(0)}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+
+box.space.test.index.pk:drop()
+ | ---
+ | ...
+
+_ = box.space.test:create_index('pk', {parts={1, 'number'}})
+ | ---
+ | ...
+
+test_run:cmd('setopt delimiter ";"')
+ | ---
+ | - true
+ | ...
+for i = 0, 32 do
+ local val = (i - 16) / 8
+ if i % 2 == 1 then val = decimal.new(val) end
+ box.space.test:insert{val}
+end;
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ""');
+ | ---
+ | - true
+ | ...
+
+box.space.test:select{}
+ | ---
+ | - - [-2]
+ | - [-1.875]
+ | - [-1.75]
+ | - [-1.625]
+ | - [-1.5]
+ | - [-1.375]
+ | - [-1.25]
+ | - [-1.125]
+ | - [-1]
+ | - [-0.875]
+ | - [-0.75]
+ | - [-0.625]
+ | - [-0.5]
+ | - [-0.375]
+ | - [-0.25]
+ | - [-0.125]
+ | - [0]
+ | - [0.125]
+ | - [0.25]
+ | - [0.375]
+ | - [0.5]
+ | - [0.625]
+ | - [0.75]
+ | - [0.875]
+ | - [1]
+ | - [1.125]
+ | - [1.25]
+ | - [1.375]
+ | - [1.5]
+ | - [1.625]
+ | - [1.75]
+ | - [1.875]
+ | - [2]
+ | ...
+
+-- check duplicates
+box.space.test:insert{-2}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+box.space.test:insert{decimal.new(-2)}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+box.space.test:insert{decimal.new(-1.875)}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+box.space.test:insert{-1.875}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+
+box.space.test.index.pk:drop()
+ | ---
+ | ...
+
+_ = box.space.test:create_index('pk')
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ";"')
+ | ---
+ | - true
+ | ...
+for i = 1,10 do
+ box.space.test:insert{i, decimal.new(i/10)}
+end;
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ""');
+ | ---
+ | - true
+ | ...
+
+-- a bigger test with a secondary index this time.
+box.space.test:insert{11, 'str'}
+ | ---
+ | - [11, 'str']
+ | ...
+box.space.test:insert{12, 0.63}
+ | ---
+ | - [12, 0.63]
+ | ...
+box.space.test:insert{13, 0.57}
+ | ---
+ | - [13, 0.57]
+ | ...
+box.space.test:insert{14, 0.33}
+ | ---
+ | - [14, 0.33]
+ | ...
+box.space.test:insert{16, 0.71}
+ | ---
+ | - [16, 0.71]
+ | ...
+
+_ = box.space.test:create_index('sk', {parts={2, 'scalar'}})
+ | ---
+ | ...
+box.space.test.index.sk <http://box.space.test.index.sk/>:select{}
+ | ---
+ | - - [1, 0.1]
+ | - [2, 0.2]
+ | - [3, 0.3]
+ | - [14, 0.33]
+ | - [4, 0.4]
+ | - [5, 0.5]
+ | - [13, 0.57]
+ | - [6, 0.6]
+ | - [12, 0.63]
+ | - [7, 0.7]
+ | - [16, 0.71]
+ | - [8, 0.8]
+ | - [9, 0.9]
+ | - [10, 1]
+ | - [11, 'str']
+ | ...
+
+box.space.test:drop()
+ | ---
+ | ...
diff --git a/test/engine/decimal.test.lua b/test/engine/decimal.test.lua
new file mode 100644
index 000000000..52a300e72
--- /dev/null
+++ b/test/engine/decimal.test.lua
@@ -0,0 +1,65 @@
+env = require('test_run')
+test_run = env.new()
+engine = test_run:get_cfg('engine')
+
+decimal = require('decimal')
+ffi = require('ffi')
+
+_ = box.schema.space.create('test', {engine=engine})
+_ = box.space.test:create_index('pk', {parts={1,'decimal'}})
+
+test_run:cmd('setopt delimiter ";"')
+for i = 0,16 do
+ box.space.test:insert{decimal.new((i-8)/4)}
+end;
+test_run:cmd('setopt delimiter ""');
+
+box.space.test:select{}
+
+-- check invalid values
+box.space.test:insert{1.23}
+box.space.test:insert{'str'}
+box.space.test:insert{ffi.new('uint64_t', 0)}
+-- check duplicates
+box.space.test:insert{decimal.new(0)}
+
+box.space.test.index.pk:drop()
+
+_ = box.space.test:create_index('pk', {parts={1, 'number'}})
+
+test_run:cmd('setopt delimiter ";"')
+for i = 0, 32 do
+ local val = (i - 16) / 8
+ if i % 2 == 1 then val = decimal.new(val) end
+ box.space.test:insert{val}
+end;
+test_run:cmd('setopt delimiter ""');
+
+box.space.test:select{}
+
+-- check duplicates
+box.space.test:insert{-2}
+box.space.test:insert{decimal.new(-2)}
+box.space.test:insert{decimal.new(-1.875)}
+box.space.test:insert{-1.875}
+
+box.space.test.index.pk:drop()
+
+_ = box.space.test:create_index('pk')
+test_run:cmd('setopt delimiter ";"')
+for i = 1,10 do
+ box.space.test:insert{i, decimal.new(i/10)}
+end;
+test_run:cmd('setopt delimiter ""');
+
+-- a bigger test with a secondary index this time.
+box.space.test:insert{11, 'str'}
+box.space.test:insert{12, 0.63}
+box.space.test:insert{13, 0.57}
+box.space.test:insert{14, 0.33}
+box.space.test:insert{16, 0.71}
+
+_ = box.space.test:create_index('sk', {parts={2, 'scalar'}})
+box.space.test.index.sk <http://box.space.test.index.sk/>:select{}
+
+box.space.test:drop()
--
2.20.1 (Apple Git-117)
>
> --
> Konstantin Osipov, Moscow, Russia
--
Serge Petrenko
sergepetrenko@tarantool.org <mailto:sergepetrenko@tarantool.org>
[-- Attachment #2: Type: text/html, Size: 121830 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [tarantool-patches] [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions
2019-08-15 8:27 ` Serge Petrenko
@ 2019-08-16 8:06 ` Konstantin Osipov
0 siblings, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-16 8:06 UTC (permalink / raw)
To: Serge Petrenko; +Cc: Vladimir Davydov, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/15 11:28]:
To be frank I don't get what's wrong with this approach.
It is not significantly slower, it is not complex either.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 5/8] box: rework field_def and tuple_compare to work with mp_field_type instead of mp_type
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
` (3 preceding siblings ...)
2019-08-08 11:55 ` [PATCH v2 4/8] lua: rework luaL_field types to support msgpack extensions Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:28 ` Konstantin Osipov
2019-08-08 11:55 ` [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack Serge Petrenko
` (2 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
This a preparation for adding decimal fields and decimal comparators. It
will be needed, since newly introduced user types, such as decimal, will
all share a single mp_type, and we want to avoid additional checks for
MP_EXT subtypes.
Prerequisite #4333
---
src/box/field_def.c | 24 ++---
src/box/field_def.h | 12 ++-
src/box/key_def.h | 2 +-
src/box/tuple_compare.cc | 228 ++++++++++++++++++++-------------------
src/box/tuple_format.c | 2 +-
5 files changed, 136 insertions(+), 132 deletions(-)
diff --git a/src/box/field_def.c b/src/box/field_def.c
index 346042b98..2fad81d42 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -49,18 +49,18 @@ const char *mp_type_strs[] = {
const uint32_t field_mp_type[] = {
/* [FIELD_TYPE_ANY] = */ UINT32_MAX,
- /* [FIELD_TYPE_UNSIGNED] = */ 1U << MP_UINT,
- /* [FIELD_TYPE_STRING] = */ 1U << MP_STR,
- /* [FIELD_TYPE_NUMBER] = */ (1U << MP_UINT) | (1U << MP_INT) |
- (1U << MP_FLOAT) | (1U << MP_DOUBLE),
- /* [FIELD_TYPE_INTEGER] = */ (1U << MP_UINT) | (1U << MP_INT),
- /* [FIELD_TYPE_BOOLEAN] = */ 1U << MP_BOOL,
- /* [FIELD_TYPE_VARBINARY] = */ 1U << MP_BIN,
- /* [FIELD_TYPE_SCALAR] = */ (1U << MP_UINT) | (1U << MP_INT) |
- (1U << MP_FLOAT) | (1U << MP_DOUBLE) | (1U << MP_STR) |
- (1U << MP_BIN) | (1U << MP_BOOL),
- /* [FIELD_TYPE_ARRAY] = */ 1U << MP_ARRAY,
- /* [FIELD_TYPE_MAP] = */ (1U << MP_MAP),
+ /* [FIELD_TYPE_UNSIGNED] = */ 1U << MP_FIELD_UINT,
+ /* [FIELD_TYPE_STRING] = */ 1U << MP_FIELD_STR,
+ /* [FIELD_TYPE_NUMBER] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT) |
+ (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE),
+ /* [FIELD_TYPE_INTEGER] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT),
+ /* [FIELD_TYPE_BOOLEAN] = */ 1U << MP_FIELD_BOOL,
+ /* [FIELD_TYPE_VARBINARY] = */ 1U << MP_FIELD_BIN,
+ /* [FIELD_TYPE_SCALAR] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT) |
+ (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE) | (1U << MP_FIELD_STR) |
+ (1U << MP_FIELD_BIN) | (1U << MP_FIELD_BOOL),
+ /* [FIELD_TYPE_ARRAY] = */ 1U << MP_FIELD_ARRAY,
+ /* [FIELD_TYPE_MAP] = */ (1U << MP_FIELD_MAP),
};
const char *field_type_strs[] = {
diff --git a/src/box/field_def.h b/src/box/field_def.h
index c1a7ec0a9..2f21f3cfc 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -37,6 +37,7 @@
#include <limits.h>
#include <msgpuck.h>
#include "opt_def.h"
+#include "mp_user_types.h"
#if defined(__cplusplus)
extern "C" {
@@ -142,15 +143,16 @@ struct field_def {
struct Expr *default_value_expr;
};
-/** Checks if mp_type (MsgPack) is compatible with field type. */
+/** Checks if mp_typeof(data) is compatible with field type. */
static inline bool
-field_mp_type_is_compatible(enum field_type type, enum mp_type mp_type,
+field_mp_type_is_compatible(enum field_type type, const char *data,
bool is_nullable)
{
+ enum mp_field_type check_type = msgpack_to_field_type(data);
assert(type < field_type_MAX);
- assert((size_t)mp_type < CHAR_BIT * sizeof(*field_mp_type));
- uint32_t mask = field_mp_type[type] | (is_nullable * (1U << MP_NIL));
- return (mask & (1U << mp_type)) != 0;
+ assert((size_t)check_type < CHAR_BIT * sizeof(*field_mp_type));
+ uint32_t mask = field_mp_type[type] | (is_nullable * (1U << MP_FIELD_NIL));
+ return (mask & (1U << check_type)) != 0;
}
static inline bool
diff --git a/src/box/key_def.h b/src/box/key_def.h
index c6e33e4ad..4ce842aff 100644
--- a/src/box/key_def.h
+++ b/src/box/key_def.h
@@ -528,7 +528,7 @@ static inline int
key_part_validate(enum field_type key_type, const char *key,
uint32_t field_no, bool is_nullable)
{
- if (unlikely(!field_mp_type_is_compatible(key_type, mp_typeof(*key),
+ if (unlikely(!field_mp_type_is_compatible(key_type, key,
is_nullable))) {
diag_set(ClientError, ER_KEY_PART_TYPE, field_no,
field_type_strs[key_type]);
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index b7b54e21a..b4932db6d 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -77,23 +77,24 @@ enum mp_class {
};
static enum mp_class mp_classes[] = {
- /* .MP_NIL = */ MP_CLASS_NIL,
- /* .MP_UINT = */ MP_CLASS_NUMBER,
- /* .MP_INT = */ MP_CLASS_NUMBER,
- /* .MP_STR = */ MP_CLASS_STR,
- /* .MP_BIN = */ MP_CLASS_BIN,
- /* .MP_ARRAY = */ MP_CLASS_ARRAY,
- /* .MP_MAP = */ MP_CLASS_MAP,
- /* .MP_BOOL = */ MP_CLASS_BOOL,
- /* .MP_FLOAT = */ MP_CLASS_NUMBER,
- /* .MP_DOUBLE = */ MP_CLASS_NUMBER,
- /* .MP_BIN = */ MP_CLASS_BIN
+ /* .MP_FIELD_UNKNOWN = */ mp_class_max,
+ /* .MP_FIELD_NIL = */ MP_CLASS_NIL,
+ /* .MP_FIELD_INT = */ MP_CLASS_NUMBER,
+ /* .MP_FIELD_UINT = */ MP_CLASS_NUMBER,
+ /* .MP_FIELD_STR = */ MP_CLASS_STR,
+ /* .MP_FIELD_BIN = */ MP_CLASS_BIN,
+ /* .MP_FIELD_ARRAY = */ MP_CLASS_ARRAY,
+ /* .MP_FIELD_MAP = */ MP_CLASS_MAP,
+ /* .MP_FIELD_BOOL = */ MP_CLASS_BOOL,
+ /* .MP_FIELD_FLOAT = */ MP_CLASS_NUMBER,
+ /* .MP_FIELD_DOUBLE = */ MP_CLASS_NUMBER,
+ /* .MP_FIELD_BIN = */ MP_CLASS_BIN
};
#define COMPARE_RESULT(a, b) (a < b ? -1 : a > b)
static enum mp_class
-mp_classof(enum mp_type type)
+mp_classof(enum mp_field_type type)
{
return mp_classes[type];
}
@@ -107,14 +108,14 @@ mp_compare_bool(const char *field_a, const char *field_b)
}
static int
-mp_compare_integer_with_type(const char *field_a, enum mp_type a_type,
- const char *field_b, enum mp_type b_type)
+mp_compare_integer_with_type(const char *field_a, enum mp_field_type a_type,
+ const char *field_b, enum mp_field_type b_type)
{
assert(mp_classof(a_type) == MP_CLASS_NUMBER);
assert(mp_classof(b_type) == MP_CLASS_NUMBER);
- if (a_type == MP_UINT) {
+ if (a_type == MP_FIELD_UINT) {
uint64_t a_val = mp_decode_uint(&field_a);
- if (b_type == MP_UINT) {
+ if (b_type == MP_FIELD_UINT) {
uint64_t b_val = mp_decode_uint(&field_b);
return COMPARE_RESULT(a_val, b_val);
} else {
@@ -125,7 +126,7 @@ mp_compare_integer_with_type(const char *field_a, enum mp_type a_type,
}
} else {
int64_t a_val = mp_decode_int(&field_a);
- if (b_type == MP_UINT) {
+ if (b_type == MP_FIELD_UINT) {
uint64_t b_val = mp_decode_uint(&field_b);
if (a_val < 0)
return -1;
@@ -209,10 +210,10 @@ mp_compare_double_uint64(double lhs, uint64_t rhs, int k)
}
static int
-mp_compare_double_any_int(double lhs, const char *rhs, enum mp_type rhs_type,
+mp_compare_double_any_int(double lhs, const char *rhs, enum mp_field_type rhs_type,
int k)
{
- if (rhs_type == MP_INT) {
+ if (rhs_type == MP_FIELD_INT) {
int64_t v = mp_decode_int(&rhs);
if (v < 0) {
return mp_compare_double_uint64(-lhs, (uint64_t)-v,
@@ -220,18 +221,18 @@ mp_compare_double_any_int(double lhs, const char *rhs, enum mp_type rhs_type,
}
return mp_compare_double_uint64(lhs, (uint64_t)v, k);
}
- assert(rhs_type == MP_UINT);
+ assert(rhs_type == MP_FIELD_UINT);
return mp_compare_double_uint64(lhs, mp_decode_uint(&rhs), k);
}
static int
mp_compare_double_any_number(double lhs, const char *rhs,
- enum mp_type rhs_type, int k)
+ enum mp_field_type rhs_type, int k)
{
double v;
- if (rhs_type == MP_FLOAT)
+ if (rhs_type == MP_FIELD_FLOAT)
v = mp_decode_float(&rhs);
- else if (rhs_type == MP_DOUBLE)
+ else if (rhs_type == MP_FIELD_DOUBLE)
v = mp_decode_double(&rhs);
else
return mp_compare_double_any_int(lhs, rhs, rhs_type, k);
@@ -265,42 +266,42 @@ mp_compare_double_any_number(double lhs, const char *rhs,
}
static int
-mp_compare_number_with_type(const char *lhs, enum mp_type lhs_type,
- const char *rhs, enum mp_type rhs_type)
+mp_compare_number_with_type(const char *lhs, enum mp_field_type lhs_type,
+ const char *rhs, enum mp_field_type rhs_type)
{
assert(mp_classof(lhs_type) == MP_CLASS_NUMBER);
assert(mp_classof(rhs_type) == MP_CLASS_NUMBER);
- if (rhs_type == MP_FLOAT) {
+ if (rhs_type == MP_FIELD_FLOAT) {
return mp_compare_double_any_number(
mp_decode_float(&rhs), lhs, lhs_type, -1
);
}
- if (rhs_type == MP_DOUBLE) {
+ if (rhs_type == MP_FIELD_DOUBLE) {
return mp_compare_double_any_number(
mp_decode_double(&rhs), lhs, lhs_type, -1
);
}
- assert(rhs_type == MP_INT || rhs_type == MP_UINT);
- if (lhs_type == MP_FLOAT) {
+ assert(rhs_type == MP_FIELD_INT || rhs_type == MP_FIELD_UINT);
+ if (lhs_type == MP_FIELD_FLOAT) {
return mp_compare_double_any_int(
mp_decode_float(&lhs), rhs, rhs_type, 1
);
}
- if (lhs_type == MP_DOUBLE) {
+ if (lhs_type == MP_FIELD_DOUBLE) {
return mp_compare_double_any_int(
mp_decode_double(&lhs), rhs, rhs_type, 1
);
}
- assert(lhs_type == MP_INT || lhs_type == MP_UINT);
+ assert(lhs_type == MP_FIELD_INT || lhs_type == MP_FIELD_UINT);
return mp_compare_integer_with_type(lhs, lhs_type, rhs, rhs_type);
}
static inline int
mp_compare_number(const char *lhs, const char *rhs)
{
- return mp_compare_number_with_type(lhs, mp_typeof(*lhs),
- rhs, mp_typeof(*rhs));
+ return mp_compare_number_with_type(lhs, msgpack_to_field_type(lhs),
+ rhs, msgpack_to_field_type(rhs));
}
static inline int
@@ -345,9 +346,10 @@ static mp_compare_f mp_class_comparators[] = {
};
static int
-mp_compare_scalar_with_type(const char *field_a, enum mp_type a_type,
- const char *field_b, enum mp_type b_type)
+mp_compare_scalar_with_type(const char *field_a, enum mp_field_type a_type,
+ const char *field_b, enum mp_field_type b_type)
{
+ assert(a_type != MP_FIELD_UNKNOWN && b_type != MP_FIELD_UNKNOWN);
enum mp_class a_class = mp_classof(a_type);
enum mp_class b_class = mp_classof(b_type);
if (a_class != b_class)
@@ -360,17 +362,17 @@ mp_compare_scalar_with_type(const char *field_a, enum mp_type a_type,
static inline int
mp_compare_scalar(const char *field_a, const char *field_b)
{
- return mp_compare_scalar_with_type(field_a, mp_typeof(*field_a),
- field_b, mp_typeof(*field_b));
+ return mp_compare_scalar_with_type(field_a, msgpack_to_field_type(field_a),
+ field_b, msgpack_to_field_type(field_b));
}
static inline int
mp_compare_scalar_coll(const char *field_a, const char *field_b,
struct coll *coll)
{
- enum mp_type type_a = mp_typeof(*field_a);
- enum mp_type type_b = mp_typeof(*field_b);
- if (type_a == MP_STR && type_b == MP_STR)
+ enum mp_field_type type_a = msgpack_to_field_type(field_a);
+ enum mp_field_type type_b = msgpack_to_field_type(field_b);
+ if (type_a == MP_FIELD_STR && type_b == MP_FIELD_STR)
return mp_compare_str_coll(field_a, field_b, coll);
return mp_compare_scalar_with_type(field_a, type_a, field_b, type_b);
}
@@ -397,9 +399,9 @@ tuple_compare_field(const char *field_a, const char *field_b,
mp_compare_str(field_a, field_b);
case FIELD_TYPE_INTEGER:
return mp_compare_integer_with_type(field_a,
- mp_typeof(*field_a),
+ msgpack_to_field_type(field_a),
field_b,
- mp_typeof(*field_b));
+ msgpack_to_field_type(field_b));
case FIELD_TYPE_NUMBER:
return mp_compare_number(field_a, field_b);
case FIELD_TYPE_BOOLEAN:
@@ -417,8 +419,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
}
static int
-tuple_compare_field_with_type(const char *field_a, enum mp_type a_type,
- const char *field_b, enum mp_type b_type,
+tuple_compare_field_with_type(const char *field_a, enum mp_field_type a_type,
+ const char *field_b, enum mp_field_type b_type,
int8_t type, struct coll *coll)
{
switch (type) {
@@ -482,11 +484,11 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
return tuple_compare_field(tuple_a_raw, tuple_b_raw,
part->type, part->coll);
}
- enum mp_type a_type = mp_typeof(*tuple_a_raw);
- enum mp_type b_type = mp_typeof(*tuple_b_raw);
- if (a_type == MP_NIL)
- return b_type == MP_NIL ? 0 : -1;
- else if (b_type == MP_NIL)
+ enum mp_field_type a_type = msgpack_to_field_type(tuple_a_raw);
+ enum mp_field_type b_type = msgpack_to_field_type(tuple_b_raw);
+ if (a_type == MP_FIELD_NIL)
+ return b_type == MP_FIELD_NIL ? 0 : -1;
+ else if (b_type == MP_FIELD_NIL)
return 1;
return tuple_compare_field_with_type(tuple_a_raw, a_type,
tuple_b_raw, b_type,
@@ -500,7 +502,7 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
const uint32_t *field_map_b = tuple_field_map(tuple_b);
struct key_part *end;
const char *field_a, *field_b;
- enum mp_type a_type, b_type;
+ enum mp_field_type a_type, b_type;
if (is_nullable)
end = part + key_def->unique_part_count;
else
@@ -538,17 +540,17 @@ tuple_compare_slowpath(struct tuple *tuple_a, hint_t tuple_a_hint,
continue;
}
if (has_optional_parts) {
- a_type = field_a != NULL ? mp_typeof(*field_a) : MP_NIL;
- b_type = field_b != NULL ? mp_typeof(*field_b) : MP_NIL;
+ a_type = field_a != NULL ? msgpack_to_field_type(field_a) : MP_FIELD_NIL;
+ b_type = field_b != NULL ? msgpack_to_field_type(field_b) : MP_FIELD_NIL;
} else {
- a_type = mp_typeof(*field_a);
- b_type = mp_typeof(*field_b);
+ a_type = msgpack_to_field_type(field_a);
+ b_type = msgpack_to_field_type(field_b);
}
- if (a_type == MP_NIL) {
- if (b_type != MP_NIL)
+ if (a_type == MP_FIELD_NIL) {
+ if (b_type != MP_FIELD_NIL)
return -1;
was_null_met = true;
- } else if (b_type == MP_NIL) {
+ } else if (b_type == MP_FIELD_NIL) {
return 1;
} else {
rc = tuple_compare_field_with_type(field_a, a_type,
@@ -629,7 +631,7 @@ tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
struct tuple_format *format = tuple_format(tuple);
const char *tuple_raw = tuple_data(tuple);
const uint32_t *field_map = tuple_field_map(tuple);
- enum mp_type a_type, b_type;
+ enum mp_field_type a_type, b_type;
if (likely(part_count == 1)) {
const char *field;
if (is_multikey) {
@@ -649,13 +651,13 @@ tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
part->coll);
}
if (has_optional_parts)
- a_type = field != NULL ? mp_typeof(*field) : MP_NIL;
+ a_type = field != NULL ? msgpack_to_field_type(field) : MP_FIELD_NIL;
else
- a_type = mp_typeof(*field);
- b_type = mp_typeof(*key);
- if (a_type == MP_NIL) {
- return b_type == MP_NIL ? 0 : -1;
- } else if (b_type == MP_NIL) {
+ a_type = msgpack_to_field_type(field);
+ b_type = msgpack_to_field_type(key);
+ if (a_type == MP_FIELD_NIL) {
+ return b_type == MP_FIELD_NIL ? 0 : -1;
+ } else if (b_type == MP_FIELD_NIL) {
return 1;
} else {
return tuple_compare_field_with_type(field, a_type, key,
@@ -688,14 +690,14 @@ tuple_compare_with_key_slowpath(struct tuple *tuple, hint_t tuple_hint,
continue;
}
if (has_optional_parts)
- a_type = field != NULL ? mp_typeof(*field) : MP_NIL;
+ a_type = field != NULL ? msgpack_to_field_type(field) : MP_FIELD_NIL;
else
- a_type = mp_typeof(*field);
- b_type = mp_typeof(*key);
- if (a_type == MP_NIL) {
- if (b_type != MP_NIL)
+ a_type = msgpack_to_field_type(field);
+ b_type = msgpack_to_field_type(key);
+ if (a_type == MP_FIELD_NIL) {
+ if (b_type != MP_FIELD_NIL)
return -1;
- } else if (b_type == MP_NIL) {
+ } else if (b_type == MP_FIELD_NIL) {
return 1;
} else {
rc = tuple_compare_field_with_type(field, a_type, key,
@@ -721,11 +723,11 @@ key_compare_parts(const char *key_a, const char *key_b, uint32_t part_count,
return tuple_compare_field(key_a, key_b, part->type,
part->coll);
}
- enum mp_type a_type = mp_typeof(*key_a);
- enum mp_type b_type = mp_typeof(*key_b);
- if (a_type == MP_NIL) {
- return b_type == MP_NIL ? 0 : -1;
- } else if (b_type == MP_NIL) {
+ enum mp_field_type a_type = msgpack_to_field_type(key_a);
+ enum mp_field_type b_type = msgpack_to_field_type(key_b);
+ if (a_type == MP_FIELD_NIL) {
+ return b_type == MP_FIELD_NIL ? 0 : -1;
+ } else if (b_type == MP_FIELD_NIL) {
return 1;
} else {
return tuple_compare_field_with_type(key_a, a_type,
@@ -746,12 +748,12 @@ key_compare_parts(const char *key_a, const char *key_b, uint32_t part_count,
else
continue;
}
- enum mp_type a_type = mp_typeof(*key_a);
- enum mp_type b_type = mp_typeof(*key_b);
- if (a_type == MP_NIL) {
- if (b_type != MP_NIL)
+ enum mp_field_type a_type = msgpack_to_field_type(key_a);
+ enum mp_field_type b_type = msgpack_to_field_type(key_b);
+ if (a_type == MP_FIELD_NIL) {
+ if (b_type != MP_FIELD_NIL)
return -1;
- } else if (b_type == MP_NIL) {
+ } else if (b_type == MP_FIELD_NIL) {
return 1;
} else {
rc = tuple_compare_field_with_type(key_a, a_type, key_b,
@@ -802,7 +804,7 @@ tuple_compare_with_key_sequential(struct tuple *tuple, hint_t tuple_hint,
key += tuple->bsize - mp_sizeof_array(field_count);
for (uint32_t i = field_count; i < part_count;
++i, mp_next(&key)) {
- if (mp_typeof(*key) != MP_NIL)
+ if (msgpack_to_field_type(key) != MP_FIELD_NIL)
return -1;
}
}
@@ -859,19 +861,19 @@ tuple_compare_sequential(struct tuple *tuple_a, hint_t tuple_a_hint,
struct key_part *end = part + key_def->unique_part_count;
uint32_t i = 0;
for (; part < end; ++part, ++i) {
- enum mp_type a_type, b_type;
+ enum mp_field_type a_type, b_type;
if (has_optional_parts) {
- a_type = i >= fc_a ? MP_NIL : mp_typeof(*key_a);
- b_type = i >= fc_b ? MP_NIL : mp_typeof(*key_b);
+ a_type = i >= fc_a ? MP_FIELD_NIL : msgpack_to_field_type(key_a);
+ b_type = i >= fc_b ? MP_FIELD_NIL : msgpack_to_field_type(key_b);
} else {
- a_type = mp_typeof(*key_a);
- b_type = mp_typeof(*key_b);
+ a_type = msgpack_to_field_type(key_a);
+ b_type = msgpack_to_field_type(key_b);
}
- if (a_type == MP_NIL) {
- if (b_type != MP_NIL)
+ if (a_type == MP_FIELD_NIL) {
+ if (b_type != MP_FIELD_NIL)
return -1;
was_null_met = true;
- } else if (b_type == MP_NIL) {
+ } else if (b_type == MP_FIELD_NIL) {
return 1;
} else {
rc = tuple_compare_field_with_type(key_a, a_type, key_b,
@@ -1292,9 +1294,9 @@ func_index_compare(struct tuple *tuple_a, hint_t tuple_a_hint,
const char *key_a = (const char *)tuple_a_hint;
const char *key_b = (const char *)tuple_b_hint;
- assert(mp_typeof(*key_a) == MP_ARRAY);
+ assert(msgpack_to_field_type(key_a) == MP_FIELD_ARRAY);
uint32_t part_count_a = mp_decode_array(&key_a);
- assert(mp_typeof(*key_b) == MP_ARRAY);
+ assert(msgpack_to_field_type(key_b) == MP_FIELD_ARRAY);
uint32_t part_count_b = mp_decode_array(&key_b);
uint32_t key_part_count = MIN(part_count_a, part_count_b);
@@ -1349,7 +1351,7 @@ func_index_compare_with_key(struct tuple *tuple, hint_t tuple_hint,
assert(key_def->for_func_index);
assert(is_nullable == key_def->is_nullable);
const char *tuple_key = (const char *)tuple_hint;
- assert(mp_typeof(*tuple_key) == MP_ARRAY);
+ assert(msgpack_to_field_type(tuple_key) == MP_FIELD_ARRAY);
uint32_t tuple_key_count = mp_decode_array(&tuple_key);
part_count = MIN(part_count, tuple_key_count);
@@ -1541,24 +1543,24 @@ hint_bin(const char *s, uint32_t len)
static inline hint_t
field_hint_boolean(const char *field)
{
- assert(mp_typeof(*field) == MP_BOOL);
+ assert(msgpack_to_field_type(field) == MP_FIELD_BOOL);
return hint_bool(mp_decode_bool(&field));
}
static inline hint_t
field_hint_unsigned(const char *field)
{
- assert(mp_typeof(*field) == MP_UINT);
+ assert(msgpack_to_field_type(field) == MP_FIELD_UINT);
return hint_uint(mp_decode_uint(&field));
}
static inline hint_t
field_hint_integer(const char *field)
{
- switch (mp_typeof(*field)) {
- case MP_UINT:
+ switch (msgpack_to_field_type(field)) {
+ case MP_FIELD_UINT:
return hint_uint(mp_decode_uint(&field));
- case MP_INT:
+ case MP_FIELD_INT:
return hint_int(mp_decode_int(&field));
default:
unreachable();
@@ -1569,14 +1571,14 @@ field_hint_integer(const char *field)
static inline hint_t
field_hint_number(const char *field)
{
- switch (mp_typeof(*field)) {
- case MP_UINT:
+ switch (msgpack_to_field_type(field)) {
+ case MP_FIELD_UINT:
return hint_uint(mp_decode_uint(&field));
- case MP_INT:
+ case MP_FIELD_INT:
return hint_int(mp_decode_int(&field));
- case MP_FLOAT:
+ case MP_FIELD_FLOAT:
return hint_double(mp_decode_float(&field));
- case MP_DOUBLE:
+ case MP_FIELD_DOUBLE:
return hint_double(mp_decode_double(&field));
default:
unreachable();
@@ -1587,7 +1589,7 @@ field_hint_number(const char *field)
static inline hint_t
field_hint_string(const char *field, struct coll *coll)
{
- assert(mp_typeof(*field) == MP_STR);
+ assert(msgpack_to_field_type(field) == MP_FIELD_STR);
uint32_t len = mp_decode_strl(&field);
return coll == NULL ? hint_str(field, len) :
hint_str_coll(field, len, coll);
@@ -1596,7 +1598,7 @@ field_hint_string(const char *field, struct coll *coll)
static inline hint_t
field_hint_varbinary(const char *field)
{
- assert(mp_typeof(*field) == MP_BIN);
+ assert(msgpack_to_field_type(field) == MP_FIELD_BIN);
uint32_t len = mp_decode_binl(&field);
return hint_bin(field, len);
}
@@ -1605,22 +1607,22 @@ static inline hint_t
field_hint_scalar(const char *field, struct coll *coll)
{
uint32_t len;
- switch(mp_typeof(*field)) {
- case MP_BOOL:
+ switch(msgpack_to_field_type(field)) {
+ case MP_FIELD_BOOL:
return hint_bool(mp_decode_bool(&field));
- case MP_UINT:
+ case MP_FIELD_UINT:
return hint_uint(mp_decode_uint(&field));
- case MP_INT:
+ case MP_FIELD_INT:
return hint_int(mp_decode_int(&field));
- case MP_FLOAT:
+ case MP_FIELD_FLOAT:
return hint_double(mp_decode_float(&field));
- case MP_DOUBLE:
+ case MP_FIELD_DOUBLE:
return hint_double(mp_decode_double(&field));
- case MP_STR:
+ case MP_FIELD_STR:
len = mp_decode_strl(&field);
return coll == NULL ? hint_str(field, len) :
hint_str_coll(field, len, coll);
- case MP_BIN:
+ case MP_FIELD_BIN:
len = mp_decode_binl(&field);
return hint_bin(field, len);
default:
@@ -1633,7 +1635,7 @@ template <enum field_type type, bool is_nullable>
static inline hint_t
field_hint(const char *field, struct coll *coll)
{
- if (is_nullable && mp_typeof(*field) == MP_NIL)
+ if (is_nullable && msgpack_to_field_type(field) == MP_FIELD_NIL)
return hint_nil();
switch (type) {
case FIELD_TYPE_BOOLEAN:
diff --git a/src/box/tuple_format.c b/src/box/tuple_format.c
index 514d5d9c0..bd7f985db 100644
--- a/src/box/tuple_format.c
+++ b/src/box/tuple_format.c
@@ -1169,7 +1169,7 @@ tuple_format_iterator_next(struct tuple_format_iterator *it,
* defined in format.
*/
bool is_nullable = tuple_field_is_nullable(field);
- if (!field_mp_type_is_compatible(field->type, mp_typeof(*entry->data),
+ if (!field_mp_type_is_compatible(field->type, entry->data,
is_nullable) != 0) {
diag_set(ClientError, ER_FIELD_TYPE,
tuple_field_path(field),
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 5/8] box: rework field_def and tuple_compare to work with mp_field_type instead of mp_type
2019-08-08 11:55 ` [PATCH v2 5/8] box: rework field_def and tuple_compare to work with mp_field_type instead of mp_type Serge Petrenko
@ 2019-08-12 21:28 ` Konstantin Osipov
0 siblings, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:28 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
> This a preparation for adding decimal fields and decimal comparators. It
> will be needed, since newly introduced user types, such as decimal, will
> all share a single mp_type, and we want to avoid additional checks for
> MP_EXT subtypes.
I think this is confusing as hell. This new enum doesn't represent
anything, it's neither field_type (otherwise it would be enum
field_type) nor mp_type.
Sounds like it is a strange "weight" or "code point" calculated for
a Cartesian product of field_type, mp_type and mp_ext type. Well,
if you want to optimize the code to look at this code point, let's
at least make it clear that it's not a type, but an optimization
technique we use.
Or let's just look at a combination of field_type and mp_type and
not introduce this strange entity just to preserve a few switches
around.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
` (4 preceding siblings ...)
2019-08-08 11:55 ` [PATCH v2 5/8] box: rework field_def and tuple_compare to work with mp_field_type instead of mp_type Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:29 ` Konstantin Osipov
2019-08-12 21:34 ` Konstantin Osipov
2019-08-08 11:55 ` [PATCH v2 7/8] decimal: add conversions to (u)int64_t Serge Petrenko
2019-08-08 11:55 ` [PATCH v2 8/8] decimal: allow to index decimals Serge Petrenko
7 siblings, 2 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
This patch adds the methods necessary to encode and decode decimals to
MsgPack. MsgPack EXT type (MP_EXT) together with a new extension type
MP_DECIMAL is used as a record header.
The decimal MsgPack representation looks like this:
+--------+-------------------+------------+===============+
| MP_EXT | length (optional) | MP_DECIMAL | PackedDecimal |
+--------+-------------------+------------+===============+
The whole record may be encoded and decoded with
mp_encode_decimal() and mp_decode_decimal(). This is equivalent to
performing mp_encode_extl()/mp_decode_extl() on the first 3 fields and
decimal_pack/unpack() on the PackedDecimal field.
It is also possible to decode and encode decimals to msgpack from lua,
which means you can insert decimals into spaces, but only into unindexed
fields for now.
Follow up #692
Part of #4333
---
| 4 +
src/lib/core/CMakeLists.txt | 1 +
src/lib/core/mp_decimal.c | 72 ++++++++++
src/lib/core/mp_decimal.h | 62 ++++++++
src/lib/core/mp_user_types.h | 19 ++-
src/lib/core/mpstream.c | 11 ++
src/lib/core/mpstream.h | 4 +
src/lua/decimal.c | 4 +-
src/lua/decimal.h | 7 +
src/lua/msgpack.c | 18 +++
src/lua/msgpackffi.lua | 36 +++++
src/lua/utils.c | 7 +
src/lua/utils.h | 2 +
test/app-tap/lua/serializer_test.lua | 14 ++
test/app-tap/msgpackffi.test.lua | 3 +-
test/app/decimal.result | 128 ++++++++---------
test/app/msgpack.result | 41 ++++++
test/app/msgpack.test.lua | 15 ++
test/box/tuple.result | 85 +++++++++++
test/box/tuple.test.lua | 23 +++
test/engine/decimal.result | 81 +++++++++++
test/engine/decimal.test.lua | 21 +++
test/unit/decimal.c | 60 +++++++-
test/unit/decimal.result | 202 ++++++++++++++++++++++++++-
third_party/lua-cjson/lua_cjson.c | 5 +
third_party/lua-yaml/lyaml.cc | 5 +
26 files changed, 859 insertions(+), 71 deletions(-)
create mode 100644 src/lib/core/mp_decimal.c
create mode 100644 src/lib/core/mp_decimal.h
create mode 100644 test/engine/decimal.result
create mode 100644 test/engine/decimal.test.lua
--git a/extra/exports b/extra/exports
index b8c42c0df..7b84a1452 100644
--- a/extra/exports
+++ b/extra/exports
@@ -62,8 +62,12 @@ PMurHash32_Result
crc32_calc
mp_encode_double
mp_encode_float
+mp_encode_decimal
mp_decode_double
mp_decode_float
+mp_decode_extl
+mp_sizeof_decimal
+decimal_unpack
log_type
say_set_log_level
diff --git a/src/lib/core/CMakeLists.txt b/src/lib/core/CMakeLists.txt
index 4b96d1105..e60b5199e 100644
--- a/src/lib/core/CMakeLists.txt
+++ b/src/lib/core/CMakeLists.txt
@@ -27,6 +27,7 @@ set(core_sources
mpstream.c
port.c
decimal.c
+ mp_decimal.c
)
if (TARGET_OS_NETBSD)
diff --git a/src/lib/core/mp_decimal.c b/src/lib/core/mp_decimal.c
new file mode 100644
index 000000000..1bcf316fa
--- /dev/null
+++ b/src/lib/core/mp_decimal.c
@@ -0,0 +1,72 @@
+/*
+ * Copyright 2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include "mp_decimal.h"
+#include "mp_user_types.h"
+#include "msgpuck.h"
+#include "decimal.h"
+
+uint32_t
+mp_sizeof_decimal(const decimal_t *dec)
+{
+ return mp_sizeof_ext(decimal_len(dec));
+}
+
+decimal_t *
+mp_decode_decimal(const char **data, decimal_t *dec)
+{
+ if (mp_typeof(**data) != MP_EXT)
+ return NULL;
+
+ int8_t type;
+ uint32_t len;
+ const char *const svp = *data;
+
+ len = mp_decode_extl(data, &type);
+
+ if (type != MP_DECIMAL || len == 0) {
+ *data = svp;
+ return NULL;
+ }
+ decimal_t *res = decimal_unpack(data, len, dec);
+ if (!res)
+ *data = svp;
+ return res;
+}
+
+char *
+mp_encode_decimal(char *data, const decimal_t *dec)
+{
+ uint32_t len = decimal_len(dec);
+ data = mp_encode_extl(data, MP_DECIMAL, len);
+ data = decimal_pack(data, dec);
+ return data;
+}
diff --git a/src/lib/core/mp_decimal.h b/src/lib/core/mp_decimal.h
new file mode 100644
index 000000000..a991a5f16
--- /dev/null
+++ b/src/lib/core/mp_decimal.h
@@ -0,0 +1,62 @@
+#ifndef TARANTOOL_LIB_CORE_MP_DECIMAL_INCLUDED
+#define TARANTOOL_LIB_CORE_MP_DECIMAL_INCLUDED
+/*
+ * Copyright 2019, Tarantool AUTHORS, please see AUTHORS file.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the
+ * following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY <COPYRIGHT HOLDER> ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
+ * <COPYRIGHT HOLDER> OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include "decimal.h"
+#include <stdint.h>
+
+/**
+ * \brief Calculate exact buffer size needed to store a decimal
+ * pointed to by \a dec.
+ */
+uint32_t
+mp_sizeof_decimal(const decimal_t *dec);
+
+/**
+ * \brief Decode a decimal from MsgPack \a data
+ * \param data - buffer pointer
+ * \return the decoded decimal
+ * \post *data = *data + mp_sizeof_decimal(retval)
+ */
+decimal_t *
+mp_decode_decimal(const char **data, decimal_t *dec);
+
+/**
+ * \brief Encode a decimal pointed to by \a dec.
+ * \parad dec - decimal pointer
+ * \param data - a buffer
+ * \return \a data + mp_sizeof_decimal(\a dec)
+ */
+char *
+mp_encode_decimal(char *data, const decimal_t *dec);
+
+#endif
diff --git a/src/lib/core/mp_user_types.h b/src/lib/core/mp_user_types.h
index d84748b71..89ea26b41 100644
--- a/src/lib/core/mp_user_types.h
+++ b/src/lib/core/mp_user_types.h
@@ -31,6 +31,12 @@
* SUCH DAMAGE.
*/
+#include "msgpuck.h"
+
+enum mp_user_type {
+ MP_DECIMAL = 0
+};
+
enum mp_field_type {
MP_FIELD_UNKNOWN,
MP_FIELD_NIL,
@@ -43,6 +49,7 @@ enum mp_field_type {
MP_FIELD_BOOL,
MP_FIELD_FLOAT,
MP_FIELD_DOUBLE,
+ MP_FIELD_DECIMAL
};
static inline enum mp_type
@@ -59,7 +66,9 @@ field_type_to_msgpack(enum mp_field_type type)
case MP_FIELD_BOOL: return MP_BOOL;
case MP_FIELD_FLOAT: return MP_FLOAT;
case MP_FIELD_DOUBLE: return MP_DOUBLE;
- default: return MP_EXT;
+ case MP_FIELD_DECIMAL:
+ default:
+ return MP_EXT;
}
}
@@ -79,6 +88,14 @@ msgpack_to_field_type(const char *data)
case MP_FLOAT: return MP_FIELD_FLOAT;
case MP_DOUBLE: return MP_FIELD_DOUBLE;
case MP_EXT:
+ {
+ int8_t type;
+ mp_decode_extl(&data, &type);
+ if (type == MP_DECIMAL) {
+ return MP_FIELD_DECIMAL;
+ }
+ /* fallthrough */
+ }
default:
return MP_FIELD_UNKNOWN;
}
diff --git a/src/lib/core/mpstream.c b/src/lib/core/mpstream.c
index 8b7276ab1..2be1797d0 100644
--- a/src/lib/core/mpstream.c
+++ b/src/lib/core/mpstream.c
@@ -33,6 +33,7 @@
#include <assert.h>
#include <stdint.h>
#include "msgpuck.h"
+#include "mp_decimal.h"
void
mpstream_reserve_slow(struct mpstream *stream, size_t size)
@@ -186,6 +187,16 @@ mpstream_encode_binl(struct mpstream *stream, uint32_t len)
mpstream_advance(stream, pos - data);
}
+void
+mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val)
+{
+ char *data = mpstream_reserve(stream, mp_sizeof_decimal(val));
+ if (data == NULL)
+ return;
+ char *pos = mp_encode_decimal(data, val);
+ mpstream_advance(stream, pos - data);
+}
+
void
mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n)
{
diff --git a/src/lib/core/mpstream.h b/src/lib/core/mpstream.h
index 69fa76f7f..3a022daa0 100644
--- a/src/lib/core/mpstream.h
+++ b/src/lib/core/mpstream.h
@@ -32,6 +32,7 @@
*/
#include "diag.h"
+#include "decimal.h"
#if defined(__cplusplus)
extern "C" {
@@ -136,6 +137,9 @@ mpstream_encode_bool(struct mpstream *stream, bool val);
void
mpstream_encode_binl(struct mpstream *stream, uint32_t len);
+void
+mpstream_encode_decimal(struct mpstream *stream, const decimal_t *val);
+
/** Copies n bytes from memory area src to stream. */
void
mpstream_memcpy(struct mpstream *stream, const void *src, uint32_t n);
diff --git a/src/lua/decimal.c b/src/lua/decimal.c
index e480ec7be..5664e10f3 100644
--- a/src/lua/decimal.c
+++ b/src/lua/decimal.c
@@ -79,10 +79,8 @@ ldecimal_##name(struct lua_State *L) { \
return 1; \
}
-static uint32_t CTID_DECIMAL;
-
/** Push a new decimal on the stack and return a pointer to it. */
-static decimal_t *
+decimal_t *
lua_pushdecimal(struct lua_State *L)
{
decimal_t *res = luaL_pushcdata(L, CTID_DECIMAL);
diff --git a/src/lua/decimal.h b/src/lua/decimal.h
index 0485d11ef..b13e59247 100644
--- a/src/lua/decimal.h
+++ b/src/lua/decimal.h
@@ -31,12 +31,19 @@
#ifndef TARANTOOL_LUA_DECIMAL_H_INCLUDED
#define TARANTOOL_LUA_DECIMAL_H_INCLUDED
+#include "lib/core/decimal.h"
+
#if defined(__cplusplus)
extern "C" {
#endif /* defined(__cplusplus) */
+extern uint32_t CTID_DECIMAL;
+
struct lua_State;
+decimal_t *
+lua_pushdecimal(struct lua_State *L);
+
void
tarantool_lua_decimal_init(struct lua_State *L);
diff --git a/src/lua/msgpack.c b/src/lua/msgpack.c
index 6fc2b8278..2e6b5c163 100644
--- a/src/lua/msgpack.c
+++ b/src/lua/msgpack.c
@@ -41,6 +41,11 @@
#include <small/region.h>
#include <small/ibuf.h>
+#include "lua/decimal.h"
+#include "lib/core/decimal.h"
+#include "lib/core/mp_decimal.h"
+#include "lib/core/mp_user_types.h"
+
#include <fiber.h>
void
@@ -185,6 +190,9 @@ restart: /* used by MP_FIELD_UNKNOWN */
assert(field->type != MP_FIELD_UNKNOWN);
assert(lua_gettop(L) == top);
goto restart;
+ case MP_FIELD_DECIMAL:
+ mpstream_encode_decimal(stream, field->decval);
+ return MP_FIELD_DECIMAL;
}
return MP_FIELD_UNKNOWN;
}
@@ -283,6 +291,16 @@ luamp_decode(struct lua_State *L, struct luaL_serializer *cfg,
luaL_setmaphint(L, -1);
return;
}
+ case MP_FIELD_DECIMAL:
+ {
+ decimal_t *dec = lua_pushdecimal(L);
+ dec = mp_decode_decimal(data, dec);
+ if (dec == NULL) {
+ lua_pop(L, -1);
+ luaL_error(L, "msgpack.decode: invalid MsgPack");
+ }
+ return;
+ }
case MP_FIELD_UNKNOWN:
luamp_decode_extension(L, data);
break;
diff --git a/src/lua/msgpackffi.lua b/src/lua/msgpackffi.lua
index bfeedbc4b..ad6bba269 100644
--- a/src/lua/msgpackffi.lua
+++ b/src/lua/msgpackffi.lua
@@ -17,10 +17,18 @@ char *
mp_encode_float(char *data, float num);
char *
mp_encode_double(char *data, double num);
+char *
+mp_encode_decimal(char *data, decimal_t *dec);
+uint32_t
+mp_sizeof_decimal(const decimal_t *dec);
float
mp_decode_float(const char **data);
double
mp_decode_double(const char **data);
+uint32_t
+mp_decode_extl(const char **data, int8_t *type);
+decimal_t *
+decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
]])
local strict_alignment = (jit.arch == 'arm')
@@ -117,6 +125,11 @@ local function encode_double(buf, num)
builtin.mp_encode_double(p, num)
end
+local function encode_decimal(buf, num)
+ local p = buf:alloc(builtin.mp_sizeof_decimal(num))
+ builtin.mp_encode_decimal(p, num)
+end
+
local function encode_int(buf, num)
if num >= 0 then
if num <= 0x7f then
@@ -294,6 +307,7 @@ on_encode(ffi.typeof('const unsigned char'), encode_int)
on_encode(ffi.typeof('bool'), encode_bool_cdata)
on_encode(ffi.typeof('float'), encode_float)
on_encode(ffi.typeof('double'), encode_double)
+on_encode(ffi.typeof('decimal_t'), encode_decimal)
--------------------------------------------------------------------------------
-- Decoder
@@ -473,6 +487,26 @@ local function decode_map(data, size)
return setmetatable(map, msgpack.map_mt)
end
+local ext_decoder = {
+ -- MP_DECIMAL
+ [0] = function(data, len) local num = ffi.new("decimal_t") builtin.decimal_unpack(data, len, num) return num end,
+}
+
+local function decode_ext(data)
+ local t = ffi.new("int8_t[1]")
+ -- mp_decode_extl and mp_decode_decimal
+ -- need type code
+ data[0] = data[0] - 1
+ local old_data = data[0]
+ local len = builtin.mp_decode_extl(data, t)
+ local fun = ext_decoder[t[0]]
+ if type(fun) == 'function' then
+ return fun(data, len)
+ else
+ error("Unsupported extension type")
+ end
+end
+
local decoder_hint = {
--[[{{{ MP_BIN]]
[0xc4] = function(data) return decode_str(data, decode_u8(data)) end;
@@ -528,6 +562,8 @@ decode_r = function(data)
return false
elseif c == 0xc3 then
return true
+ elseif c >= 0xd4 and c <= 0xd8 or c >= 0xc7 and c <= 0xc9 then
+ return decode_ext(data)
else
local fun = decoder_hint[c];
assert (type(fun) == "function")
diff --git a/src/lua/utils.c b/src/lua/utils.c
index 9b2f4fd6c..853d735b7 100644
--- a/src/lua/utils.c
+++ b/src/lua/utils.c
@@ -45,6 +45,8 @@ static uint32_t CTID_STRUCT_IBUF;
static uint32_t CTID_STRUCT_IBUF_PTR;
uint32_t CTID_CHAR_PTR;
uint32_t CTID_CONST_CHAR_PTR;
+uint32_t CTID_DECIMAL;
+
void *
luaL_pushcdata(struct lua_State *L, uint32_t ctypeid)
@@ -722,6 +724,11 @@ luaL_tofield(struct lua_State *L, struct luaL_serializer *cfg, int index,
}
/* Fall through */
default:
+ if (cd->ctypeid == CTID_DECIMAL) {
+ field->type = MP_FIELD_DECIMAL;
+ field->decval = (decimal_t *) cdata;
+ return 0;
+ }
field->type = MP_FIELD_UNKNOWN;
}
return 0;
diff --git a/src/lua/utils.h b/src/lua/utils.h
index 4a314ab32..db9b070cd 100644
--- a/src/lua/utils.h
+++ b/src/lua/utils.h
@@ -55,6 +55,7 @@ extern "C" {
#include "lua/error.h"
#include "lib/core/mp_user_types.h"
+#include "lib/core/decimal.h" /* decimal_t */
struct lua_State;
struct ibuf;
@@ -287,6 +288,7 @@ struct luaL_field {
bool bval;
/* Array or map. */
uint32_t size;
+ decimal_t *decval;
};
enum mp_field_type type;
bool compact; /* a flag used by YAML serializer */
diff --git a/test/app-tap/lua/serializer_test.lua b/test/app-tap/lua/serializer_test.lua
index a48e557a3..e7048da4d 100644
--- a/test/app-tap/lua/serializer_test.lua
+++ b/test/app-tap/lua/serializer_test.lua
@@ -189,6 +189,17 @@ local function test_double(test, s)
ss = nil
end
+local function test_decimal(test, s)
+ local decimal = require('decimal')
+ test:plan(10)
+
+ rt(test, s, decimal.new(1), 'cdata')
+ rt(test, s, decimal.new('1e37'), 'cdata')
+ rt(test, s, decimal.new('1e-38'), 'cdata')
+ rt(test, s, decimal.new('1234567891234567890.0987654321987654321'), 'cdata')
+ rt(test, s, decimal.new('-1234567891234567890.0987654321987654321'), 'cdata')
+end
+
local function test_boolean(test, s)
test:plan(4)
@@ -223,6 +234,8 @@ local function test_nil(test, s)
test:is(t[4], s.NULL, "sparse array with NULL")
end
+
+
local function test_table(test, s, is_array, is_map)
test:plan(s.cfg and 31 or 13)
@@ -398,4 +411,5 @@ return {
test_nil = test_nil;
test_table = test_table;
test_ucdata = test_ucdata;
+ test_decimal = test_decimal;
}
diff --git a/test/app-tap/msgpackffi.test.lua b/test/app-tap/msgpackffi.test.lua
index f9646335b..f2a8f254b 100755
--- a/test/app-tap/msgpackffi.test.lua
+++ b/test/app-tap/msgpackffi.test.lua
@@ -72,10 +72,11 @@ end
tap.test("msgpackffi", function(test)
local serializer = require('msgpackffi')
- test:plan(9)
+ test:plan(10)
test:test("unsigned", common.test_unsigned, serializer)
test:test("signed", common.test_signed, serializer)
test:test("double", common.test_double, serializer)
+ test:test("decimal", common.test_decimal, serializer)
test:test("boolean", common.test_boolean, serializer)
test:test("string", common.test_string, serializer)
test:test("nil", common.test_nil, serializer)
diff --git a/test/app/decimal.result b/test/app/decimal.result
index 633f3681a..51c1f3a25 100644
--- a/test/app/decimal.result
+++ b/test/app/decimal.result
@@ -12,77 +12,77 @@ ffi = require('ffi')
-- check various constructors
decimal.new('1234.5678')
| ---
- | - '1234.5678'
+ | - 1234.5678
| ...
decimal.new('1e6')
| ---
- | - '1000000'
+ | - 1000000
| ...
decimal.new('-6.234612e2')
| ---
- | - '-623.4612'
+ | - -623.4612
| ...
-- check (u)int16/32/64_t
decimal.new(2ULL ^ 63)
| ---
- | - '9223372036854775808'
+ | - 9223372036854775808
| ...
decimal.new(123456789123456789ULL)
| ---
- | - '123456789123456789'
+ | - 123456789123456789
| ...
decimal.new(-123456789123456789LL)
| ---
- | - '-123456789123456789'
+ | - -123456789123456789
| ...
decimal.new(ffi.new('uint8_t', 231))
| ---
- | - '231'
+ | - 231
| ...
decimal.new(ffi.new('int8_t', -113))
| ---
- | - '-113'
+ | - -113
| ...
decimal.new(ffi.new('uint16_t', 65535))
| ---
- | - '65535'
+ | - 65535
| ...
decimal.new(ffi.new('int16_t', -31263))
| ---
- | - '-31263'
+ | - -31263
| ...
decimal.new(ffi.new('uint32_t', 4123123123))
| ---
- | - '4123123123'
+ | - 4123123123
| ...
decimal.new(ffi.new('int32_t', -2123123123))
| ---
- | - '-2123123123'
+ | - -2123123123
| ...
decimal.new(ffi.new('float', 128.5))
| ---
- | - '128.5'
+ | - 128.5
| ...
decimal.new(ffi.new('double', 128.5))
| ---
- | - '128.5'
+ | - 128.5
| ...
decimal.new(1)
| ---
- | - '1'
+ | - 1
| ...
decimal.new(-1)
| ---
- | - '-1'
+ | - -1
| ...
decimal.new(2^64)
| ---
- | - '18446744073709600000'
+ | - 18446744073709600000
| ...
decimal.new(2^(-20))
| ---
- | - '0.00000095367431640625'
+ | - 0.00000095367431640625
| ...
-- incorrect constructions
@@ -128,38 +128,38 @@ a = decimal.new('10')
| ...
a
| ---
- | - '10'
+ | - 10
| ...
b = decimal.new('0.1')
| ---
| ...
b
| ---
- | - '0.1'
+ | - 0.1
| ...
a + b
| ---
- | - '10.1'
+ | - 10.1
| ...
a - b
| ---
- | - '9.9'
+ | - 9.9
| ...
a * b
| ---
- | - '1.0'
+ | - 1.0
| ...
a / b
| ---
- | - '100'
+ | - 100
| ...
a ^ b
| ---
- | - '1.2589254117941672104239541063958006061'
+ | - 1.2589254117941672104239541063958006061
| ...
b ^ a
| ---
- | - '0.0000000001'
+ | - 0.0000000001
| ...
-a + -b == -(a + b)
| ---
@@ -167,11 +167,11 @@ b ^ a
| ...
a
| ---
- | - '10'
+ | - 10
| ...
b
| ---
- | - '0.1'
+ | - 0.1
| ...
a < b
@@ -216,11 +216,11 @@ a ~= b
| ...
a
| ---
- | - '10'
+ | - 10
| ...
b
| ---
- | - '0.1'
+ | - 0.1
| ...
-- check comparsion with nil
@@ -251,19 +251,19 @@ a <= nil
decimal.sqrt(a)
| ---
- | - '3.1622776601683793319988935444327185337'
+ | - 3.1622776601683793319988935444327185337
| ...
decimal.ln(a)
| ---
- | - '2.3025850929940456840179914546843642076'
+ | - 2.3025850929940456840179914546843642076
| ...
decimal.log10(a)
| ---
- | - '1'
+ | - 1
| ...
decimal.exp(a)
| ---
- | - '22026.465794806716516957900645284244366'
+ | - 22026.465794806716516957900645284244366
| ...
a == decimal.ln(decimal.exp(a))
| ---
@@ -287,7 +287,7 @@ a + -a == 0
| ...
a
| ---
- | - '10'
+ | - 10
| ...
a = decimal.new('1.1234567891234567891234567891234567891')
@@ -295,7 +295,7 @@ a = decimal.new('1.1234567891234567891234567891234567891')
| ...
a
| ---
- | - '1.1234567891234567891234567891234567891'
+ | - 1.1234567891234567891234567891234567891
| ...
decimal.precision(a)
| ---
@@ -311,7 +311,7 @@ decimal.round(a, 37) == a
| ...
a
| ---
- | - '1.1234567891234567891234567891234567891'
+ | - 1.1234567891234567891234567891234567891
| ...
a = decimal.round(a, 36)
| ---
@@ -335,19 +335,19 @@ decimal.round(a, -5) == a
| ...
decimal.round(a, 7)
| ---
- | - '1.1234568'
+ | - 1.1234568
| ...
decimal.round(a, 3)
| ---
- | - '1.123'
+ | - 1.123
| ...
decimal.round(a, 0)
| ---
- | - '1'
+ | - 1
| ...
a
| ---
- | - '1.123456789123456789123456789123456789'
+ | - 1.123456789123456789123456789123456789
| ...
decimal.ln(0)
@@ -360,7 +360,7 @@ decimal.ln(-1)
| ...
decimal.ln(1)
| ---
- | - '0'
+ | - 0
| ...
decimal.log10(0)
| ---
@@ -372,7 +372,7 @@ decimal.log10(-1)
| ...
decimal.log10(1)
| ---
- | - '0'
+ | - 0
| ...
decimal.exp(88)
| ---
@@ -380,7 +380,7 @@ decimal.exp(88)
| ...
decimal.exp(87)
| ---
- | - '60760302250568721495223289381302760753'
+ | - 60760302250568721495223289381302760753
| ...
decimal.sqrt(-5)
| ---
@@ -388,7 +388,7 @@ decimal.sqrt(-5)
| ...
decimal.sqrt(5)
| ---
- | - '2.2360679774997896964091736687312762354'
+ | - 2.2360679774997896964091736687312762354
| ...
-- various incorrect operands
@@ -434,11 +434,11 @@ a ^ 2
| ...
a ^ 1.9
| ---
- | - '1258925411794167210423954106395800606.1'
+ | - 1258925411794167210423954106395800606.1
| ...
a * '1e18'
| ---
- | - '10000000000000000000000000000000000000'
+ | - 10000000000000000000000000000000000000
| ...
a = decimal.new(string.rep('9', 38))
| ---
@@ -461,7 +461,7 @@ a + '0.5'
| ...
a + '0.4'
| ---
- | - '99999999999999999999999999999999999999'
+ | - 99999999999999999999999999999999999999
| ...
a / 0.5
| ---
@@ -477,7 +477,7 @@ a = decimal.new('-13')
| ...
a ^ 2
| ---
- | - '169'
+ | - 169
| ...
-- fractional powers are allowed only for positive numbers
a ^ 2.5
@@ -490,67 +490,67 @@ a = decimal.new('1e5')
| ...
a
| ---
- | - '100000'
+ | - 100000
| ...
decimal.trim(a)
| ---
- | - '100000'
+ | - 100000
| ...
decimal.trim(decimal.rescale(a, 10))
| ---
- | - '100000'
+ | - 100000
| ...
decimal.rescale(a, 10)
| ---
- | - '100000.0000000000'
+ | - 100000.0000000000
| ...
decimal.rescale(a, -5)
| ---
- | - '100000'
+ | - 100000
| ...
decimal.rescale(a, 0)
| ---
- | - '100000'
+ | - 100000
| ...
decimal.rescale(a, 32)
| ---
- | - '100000.00000000000000000000000000000000'
+ | - 100000.00000000000000000000000000000000
| ...
-- scale too big
decimal.rescale(a, 33)
| ---
- | - '100000'
+ | - 100000
| ...
decimal.trim(decimal.rescale(a, 10))
| ---
- | - '100000'
+ | - 100000
| ...
a = decimal.new('123.456789000000000')
| ---
| ...
a
| ---
- | - '123.456789000000000'
+ | - 123.456789000000000
| ...
decimal.trim(a)
| ---
- | - '123.456789'
+ | - 123.456789
| ...
-- check correct rounding when scale = 0
decimal.round(decimal.new(0.9), 0)
| ---
- | - '1'
+ | - 1
| ...
decimal.round(decimal.new(9.9), 0)
| ---
- | - '10'
+ | - 10
| ...
decimal.round(decimal.new(99.9), 0)
| ---
- | - '100'
+ | - 100
| ...
decimal.round(decimal.new(99.4), 0)
| ---
- | - '99'
+ | - 99
| ...
diff --git a/test/app/msgpack.result b/test/app/msgpack.result
index a67c05d38..4b5aec784 100644
--- a/test/app/msgpack.result
+++ b/test/app/msgpack.result
@@ -252,3 +252,44 @@ msgpack.decode(ffi.cast('char *', '\x04\x05\x06'), -1)
---
- error: 'msgpack.decode: size can''t be negative'
...
+--
+-- gh-4333: msgpack encode/decode decimals.
+--
+decimal = require('decimal')
+---
+...
+a = decimal.new('1e37')
+---
+...
+b = decimal.new('1e-38')
+---
+...
+c = decimal.new('1')
+---
+...
+d = decimal.new('0.1234567')
+---
+...
+e = decimal.new('123.4567')
+---
+...
+msgpack.decode(msgpack.encode(a)) == a
+---
+- true
+...
+msgpack.decode(msgpack.encode(b)) == b
+---
+- true
+...
+msgpack.decode(msgpack.encode(c)) == c
+---
+- true
+...
+msgpack.decode(msgpack.encode(d)) == d
+---
+- true
+...
+msgpack.decode(msgpack.encode(e)) == e
+---
+- true
+...
diff --git a/test/app/msgpack.test.lua b/test/app/msgpack.test.lua
index e0880ac22..9224d870a 100644
--- a/test/app/msgpack.test.lua
+++ b/test/app/msgpack.test.lua
@@ -84,3 +84,18 @@ size = msgpack.encode(100, buf)
-- is not negative.
--
msgpack.decode(ffi.cast('char *', '\x04\x05\x06'), -1)
+
+--
+-- gh-4333: msgpack encode/decode decimals.
+--
+decimal = require('decimal')
+a = decimal.new('1e37')
+b = decimal.new('1e-38')
+c = decimal.new('1')
+d = decimal.new('0.1234567')
+e = decimal.new('123.4567')
+msgpack.decode(msgpack.encode(a)) == a
+msgpack.decode(msgpack.encode(b)) == b
+msgpack.decode(msgpack.encode(c)) == c
+msgpack.decode(msgpack.encode(d)) == d
+msgpack.decode(msgpack.encode(e)) == e
diff --git a/test/box/tuple.result b/test/box/tuple.result
index 2561ebc35..a5010538d 100644
--- a/test/box/tuple.result
+++ b/test/box/tuple.result
@@ -1252,3 +1252,88 @@ s2:frommap({a="1", k="11"}):tomap({names_only = true})
s2:drop()
---
...
+-- test decimals in tuple
+dec = require('decimal')
+---
+...
+a = dec.new('1')
+---
+...
+b = dec.new('1e10')
+---
+...
+c = dec.new('1e-10')
+---
+...
+d = box.tuple.new{5, a, 6, b, 7, c, "string"}
+---
+...
+d
+---
+- [5, 1, 6, 10000000000, 7, 0.0000000001, 'string']
+...
+state, val = d:next()
+---
+...
+state
+---
+- 1
+...
+val
+---
+- 5
+...
+state, val = d:next(state)
+---
+...
+state
+---
+- 2
+...
+val
+---
+- 1
+...
+state, val = d:next(state)
+---
+...
+state, val
+---
+- 3
+- 6
+...
+d:slice(1)
+---
+- 1
+- 6
+- 10000000000
+- 7
+- 0.0000000001
+- string
+...
+d:slice(-1)
+---
+- string
+...
+d:slice(-2)
+---
+- 0.0000000001
+- string
+...
+msgpack.decode(msgpackffi.encode(d))
+---
+- [5, 1, 6, 10000000000, 7, 0.0000000001, 'string']
+- 24
+...
+d:bsize()
+---
+- 23
+...
+d:update{{'!', 3, dec.new('1234.5678')}}
+---
+- [5, 1, 1234.5678, 6, 10000000000, 7, 0.0000000001, 'string']
+...
+d:update{{'=', -1, dec.new('0.12345678910111213')}}
+---
+- [5, 1, 6, 10000000000, 7, 0.0000000001, 0.12345678910111213]
+...
diff --git a/test/box/tuple.test.lua b/test/box/tuple.test.lua
index b0a4ab173..8d4431bc6 100644
--- a/test/box/tuple.test.lua
+++ b/test/box/tuple.test.lua
@@ -426,3 +426,26 @@ s2:frommap({a="1", k="11"})
s2:frommap({a="1", k="11"}):tomap({names_only = true})
s2:drop()
+
+-- test decimals in tuple
+dec = require('decimal')
+a = dec.new('1')
+b = dec.new('1e10')
+c = dec.new('1e-10')
+d = box.tuple.new{5, a, 6, b, 7, c, "string"}
+d
+state, val = d:next()
+state
+val
+state, val = d:next(state)
+state
+val
+state, val = d:next(state)
+state, val
+d:slice(1)
+d:slice(-1)
+d:slice(-2)
+msgpack.decode(msgpackffi.encode(d))
+d:bsize()
+d:update{{'!', 3, dec.new('1234.5678')}}
+d:update{{'=', -1, dec.new('0.12345678910111213')}}
diff --git a/test/engine/decimal.result b/test/engine/decimal.result
new file mode 100644
index 000000000..f8888d7c9
--- /dev/null
+++ b/test/engine/decimal.result
@@ -0,0 +1,81 @@
+-- test-run result file version 2
+env = require('test_run')
+ | ---
+ | ...
+test_run = env.new()
+ | ---
+ | ...
+engine = test_run:get_cfg('engine')
+ | ---
+ | ...
+
+decimal = require('decimal')
+ | ---
+ | ...
+
+_ = box.schema.space.create('test', {engine=engine})
+ | ---
+ | ...
+_ = box.space.test:create_index('pk')
+ | ---
+ | ...
+box.space.test:insert{1, decimal.new(1.1)}
+ | ---
+ | - [1, 1.1]
+ | ...
+box.space.test:insert{2, decimal.new(2.2)}
+ | ---
+ | - [2, 2.2]
+ | ...
+box.space.test:insert{3, decimal.new(1.1)}
+ | ---
+ | - [3, 1.1]
+ | ...
+box.space.test:insert{4, decimal.new('1234567890123456789.9876543210987654321'), decimal.new(1.2345)}
+ | ---
+ | - [4, 1234567890123456789.9876543210987654321, 1.2345]
+ | ...
+box.space.test:select{}
+ | ---
+ | - - [1, 1.1]
+ | - [2, 2.2]
+ | - [3, 1.1]
+ | - [4, 1234567890123456789.9876543210987654321, 1.2345]
+ | ...
+a = box.space.test:get{4}
+ | ---
+ | ...
+a:next()
+ | ---
+ | - 1
+ | - 4
+ | ...
+a:next(1)
+ | ---
+ | - 2
+ | - 1234567890123456789.9876543210987654321
+ | ...
+a:next(2)
+ | ---
+ | - 3
+ | - 1.2345
+ | ...
+a:slice(-2)
+ | ---
+ | - 1234567890123456789.9876543210987654321
+ | - 1.2345
+ | ...
+box.space.test:replace{3, decimal.new(3.3)}
+ | ---
+ | - [3, 3.3]
+ | ...
+box.space.test:select{}
+ | ---
+ | - - [1, 1.1]
+ | - [2, 2.2]
+ | - [3, 3.3]
+ | - [4, 1234567890123456789.9876543210987654321, 1.2345]
+ | ...
+box.space.test:drop()
+ | ---
+ | ...
diff --git a/test/engine/decimal.test.lua b/test/engine/decimal.test.lua
new file mode 100644
index 000000000..1b14871b0
--- /dev/null
+++ b/test/engine/decimal.test.lua
@@ -0,0 +1,21 @@
+env = require('test_run')
+test_run = env.new()
+engine = test_run:get_cfg('engine')
+
+decimal = require('decimal')
+
+_ = box.schema.space.create('test', {engine=engine})
+_ = box.space.test:create_index('pk')
+box.space.test:insert{1, decimal.new(1.1)}
+box.space.test:insert{2, decimal.new(2.2)}
+box.space.test:insert{3, decimal.new(1.1)}
+box.space.test:insert{4, decimal.new('1234567890123456789.9876543210987654321'), decimal.new(1.2345)}
+box.space.test:select{}
+a = box.space.test:get{4}
+a:next()
+a:next(1)
+a:next(2)
+a:slice(-2)
+box.space.test:replace{3, decimal.new(3.3)}
+box.space.test:select{}
+box.space.test:drop()
diff --git a/test/unit/decimal.c b/test/unit/decimal.c
index b55333ed7..a70fe9a9e 100644
--- a/test/unit/decimal.c
+++ b/test/unit/decimal.c
@@ -1,5 +1,8 @@
#include "unit.h"
#include "decimal.h"
+#include "mp_decimal.h"
+#include "mp_user_types.h"
+#include "msgpuck.h"
#include <limits.h>
#include <string.h>
#include <float.h> /* DBL_DIG */
@@ -71,6 +74,32 @@
char buf[32];
+#define test_mpdec(str) ({\
+ decimal_t dec;\
+ decimal_from_string(&dec, str);\
+ uint32_t l1 = mp_sizeof_decimal(&dec);\
+ ok(l1 <= 25 && l1 >= 4, "mp_sizeof_decimal("str")");\
+ char *b1 = mp_encode_decimal(buf, &dec);\
+ is(b1, buf + l1, "mp_sizeof_decimal("str") == len(mp_encode_decimal("str"))");\
+ const char *b2 = buf;\
+ const char *b3 = buf;\
+ decimal_t d2;\
+ mp_next(&b3);\
+ is(b3, b1, "mp_next(mp_encode("str"))");\
+ mp_decode_decimal(&b2, &d2);\
+ is(b1, b2, "mp_decode(mp_encode("str") len");\
+ is(decimal_compare(&dec, &d2), 0, "mp_decode(mp_encode("str")) value");\
+ is(decimal_scale(&dec), decimal_scale(&d2), "mp_decode(mp_encode("str")) scale");\
+ is(strcmp(decimal_to_string(&d2), str), 0, "str(mp_decode(mp_encode("str"))) == "str);\
+ b2 = buf;\
+ int8_t type;\
+ uint32_t l2 = mp_decode_extl(&b2, &type);\
+ is(type, MP_DECIMAL, "mp_ext_type is MP_DECIMAL");\
+ is(&d2, decimal_unpack(&b2, l2, &d2), "decimal_unpack() after mp_decode_extl()");\
+ is(decimal_compare(&dec, &d2), 0, "decimal_unpack() after mp_decode_extl() value");\
+ is(b2, buf + l1, "decimal_unpack() after mp_decode_extl() len");\
+})
+
#define test_decpack(str) ({\
decimal_t dec;\
decimal_from_string(&dec, str);\
@@ -136,10 +165,37 @@ test_pack_unpack(void)
return check_plan();
}
+static int
+test_mp_decimal(void)
+{
+ plan(198);
+
+ test_mpdec("0");
+ test_mpdec("-0");
+ test_mpdec("1");
+ test_mpdec("-1");
+ test_mpdec("0.1");
+ test_mpdec("-0.1");
+ test_mpdec("2.718281828459045");
+ test_mpdec("-2.718281828459045");
+ test_mpdec("3.141592653589793");
+ test_mpdec("-3.141592653589793");
+ test_mpdec("1234567891234567890.0987654321987654321");
+ test_mpdec("-1234567891234567890.0987654321987654321");
+ test_mpdec("0.0000000000000000000000000000000000001");
+ test_mpdec("-0.0000000000000000000000000000000000001");
+ test_mpdec("0.00000000000000000000000000000000000001");
+ test_mpdec("-0.00000000000000000000000000000000000001");
+ test_mpdec("99999999999999999999999999999999999999");
+ test_mpdec("-99999999999999999999999999999999999999");
+
+ return check_plan();
+}
+
int
main(void)
{
- plan(279);
+ plan(280);
dectest(314, 271, uint64, uint64_t);
dectest(65535, 23456, uint64, uint64_t);
@@ -201,5 +257,7 @@ main(void)
test_pack_unpack();
+ test_mp_decimal();
+
return check_plan();
}
diff --git a/test/unit/decimal.result b/test/unit/decimal.result
index 2dd91af49..e6812a8dd 100644
--- a/test/unit/decimal.result
+++ b/test/unit/decimal.result
@@ -1,4 +1,4 @@
-1..279
+1..280
ok 1 - decimal(314)
ok 2 - decimal(271)
ok 3 - decimal(314) + decimal(271)
@@ -430,3 +430,203 @@ ok 278 - decimal_sqrt(-10) - error on wrong operands.
ok 150 - unpack malformed decimal fails
ok 151 - decode malformed decimal preserves buffer position
ok 279 - subtests
+ 1..198
+ ok 1 - mp_sizeof_decimal(0)
+ ok 2 - mp_sizeof_decimal(0) == len(mp_encode_decimal(0))
+ ok 3 - mp_next(mp_encode(0))
+ ok 4 - mp_decode(mp_encode(0) len
+ ok 5 - mp_decode(mp_encode(0)) value
+ ok 6 - mp_decode(mp_encode(0)) scale
+ ok 7 - str(mp_decode(mp_encode(0))) == 0
+ ok 8 - mp_ext_type is MP_DECIMAL
+ ok 9 - decimal_unpack() after mp_decode_extl()
+ ok 10 - decimal_unpack() after mp_decode_extl() value
+ ok 11 - decimal_unpack() after mp_decode_extl() len
+ ok 12 - mp_sizeof_decimal(-0)
+ ok 13 - mp_sizeof_decimal(-0) == len(mp_encode_decimal(-0))
+ ok 14 - mp_next(mp_encode(-0))
+ ok 15 - mp_decode(mp_encode(-0) len
+ ok 16 - mp_decode(mp_encode(-0)) value
+ ok 17 - mp_decode(mp_encode(-0)) scale
+ ok 18 - str(mp_decode(mp_encode(-0))) == -0
+ ok 19 - mp_ext_type is MP_DECIMAL
+ ok 20 - decimal_unpack() after mp_decode_extl()
+ ok 21 - decimal_unpack() after mp_decode_extl() value
+ ok 22 - decimal_unpack() after mp_decode_extl() len
+ ok 23 - mp_sizeof_decimal(1)
+ ok 24 - mp_sizeof_decimal(1) == len(mp_encode_decimal(1))
+ ok 25 - mp_next(mp_encode(1))
+ ok 26 - mp_decode(mp_encode(1) len
+ ok 27 - mp_decode(mp_encode(1)) value
+ ok 28 - mp_decode(mp_encode(1)) scale
+ ok 29 - str(mp_decode(mp_encode(1))) == 1
+ ok 30 - mp_ext_type is MP_DECIMAL
+ ok 31 - decimal_unpack() after mp_decode_extl()
+ ok 32 - decimal_unpack() after mp_decode_extl() value
+ ok 33 - decimal_unpack() after mp_decode_extl() len
+ ok 34 - mp_sizeof_decimal(-1)
+ ok 35 - mp_sizeof_decimal(-1) == len(mp_encode_decimal(-1))
+ ok 36 - mp_next(mp_encode(-1))
+ ok 37 - mp_decode(mp_encode(-1) len
+ ok 38 - mp_decode(mp_encode(-1)) value
+ ok 39 - mp_decode(mp_encode(-1)) scale
+ ok 40 - str(mp_decode(mp_encode(-1))) == -1
+ ok 41 - mp_ext_type is MP_DECIMAL
+ ok 42 - decimal_unpack() after mp_decode_extl()
+ ok 43 - decimal_unpack() after mp_decode_extl() value
+ ok 44 - decimal_unpack() after mp_decode_extl() len
+ ok 45 - mp_sizeof_decimal(0.1)
+ ok 46 - mp_sizeof_decimal(0.1) == len(mp_encode_decimal(0.1))
+ ok 47 - mp_next(mp_encode(0.1))
+ ok 48 - mp_decode(mp_encode(0.1) len
+ ok 49 - mp_decode(mp_encode(0.1)) value
+ ok 50 - mp_decode(mp_encode(0.1)) scale
+ ok 51 - str(mp_decode(mp_encode(0.1))) == 0.1
+ ok 52 - mp_ext_type is MP_DECIMAL
+ ok 53 - decimal_unpack() after mp_decode_extl()
+ ok 54 - decimal_unpack() after mp_decode_extl() value
+ ok 55 - decimal_unpack() after mp_decode_extl() len
+ ok 56 - mp_sizeof_decimal(-0.1)
+ ok 57 - mp_sizeof_decimal(-0.1) == len(mp_encode_decimal(-0.1))
+ ok 58 - mp_next(mp_encode(-0.1))
+ ok 59 - mp_decode(mp_encode(-0.1) len
+ ok 60 - mp_decode(mp_encode(-0.1)) value
+ ok 61 - mp_decode(mp_encode(-0.1)) scale
+ ok 62 - str(mp_decode(mp_encode(-0.1))) == -0.1
+ ok 63 - mp_ext_type is MP_DECIMAL
+ ok 64 - decimal_unpack() after mp_decode_extl()
+ ok 65 - decimal_unpack() after mp_decode_extl() value
+ ok 66 - decimal_unpack() after mp_decode_extl() len
+ ok 67 - mp_sizeof_decimal(2.718281828459045)
+ ok 68 - mp_sizeof_decimal(2.718281828459045) == len(mp_encode_decimal(2.718281828459045))
+ ok 69 - mp_next(mp_encode(2.718281828459045))
+ ok 70 - mp_decode(mp_encode(2.718281828459045) len
+ ok 71 - mp_decode(mp_encode(2.718281828459045)) value
+ ok 72 - mp_decode(mp_encode(2.718281828459045)) scale
+ ok 73 - str(mp_decode(mp_encode(2.718281828459045))) == 2.718281828459045
+ ok 74 - mp_ext_type is MP_DECIMAL
+ ok 75 - decimal_unpack() after mp_decode_extl()
+ ok 76 - decimal_unpack() after mp_decode_extl() value
+ ok 77 - decimal_unpack() after mp_decode_extl() len
+ ok 78 - mp_sizeof_decimal(-2.718281828459045)
+ ok 79 - mp_sizeof_decimal(-2.718281828459045) == len(mp_encode_decimal(-2.718281828459045))
+ ok 80 - mp_next(mp_encode(-2.718281828459045))
+ ok 81 - mp_decode(mp_encode(-2.718281828459045) len
+ ok 82 - mp_decode(mp_encode(-2.718281828459045)) value
+ ok 83 - mp_decode(mp_encode(-2.718281828459045)) scale
+ ok 84 - str(mp_decode(mp_encode(-2.718281828459045))) == -2.718281828459045
+ ok 85 - mp_ext_type is MP_DECIMAL
+ ok 86 - decimal_unpack() after mp_decode_extl()
+ ok 87 - decimal_unpack() after mp_decode_extl() value
+ ok 88 - decimal_unpack() after mp_decode_extl() len
+ ok 89 - mp_sizeof_decimal(3.141592653589793)
+ ok 90 - mp_sizeof_decimal(3.141592653589793) == len(mp_encode_decimal(3.141592653589793))
+ ok 91 - mp_next(mp_encode(3.141592653589793))
+ ok 92 - mp_decode(mp_encode(3.141592653589793) len
+ ok 93 - mp_decode(mp_encode(3.141592653589793)) value
+ ok 94 - mp_decode(mp_encode(3.141592653589793)) scale
+ ok 95 - str(mp_decode(mp_encode(3.141592653589793))) == 3.141592653589793
+ ok 96 - mp_ext_type is MP_DECIMAL
+ ok 97 - decimal_unpack() after mp_decode_extl()
+ ok 98 - decimal_unpack() after mp_decode_extl() value
+ ok 99 - decimal_unpack() after mp_decode_extl() len
+ ok 100 - mp_sizeof_decimal(-3.141592653589793)
+ ok 101 - mp_sizeof_decimal(-3.141592653589793) == len(mp_encode_decimal(-3.141592653589793))
+ ok 102 - mp_next(mp_encode(-3.141592653589793))
+ ok 103 - mp_decode(mp_encode(-3.141592653589793) len
+ ok 104 - mp_decode(mp_encode(-3.141592653589793)) value
+ ok 105 - mp_decode(mp_encode(-3.141592653589793)) scale
+ ok 106 - str(mp_decode(mp_encode(-3.141592653589793))) == -3.141592653589793
+ ok 107 - mp_ext_type is MP_DECIMAL
+ ok 108 - decimal_unpack() after mp_decode_extl()
+ ok 109 - decimal_unpack() after mp_decode_extl() value
+ ok 110 - decimal_unpack() after mp_decode_extl() len
+ ok 111 - mp_sizeof_decimal(1234567891234567890.0987654321987654321)
+ ok 112 - mp_sizeof_decimal(1234567891234567890.0987654321987654321) == len(mp_encode_decimal(1234567891234567890.0987654321987654321))
+ ok 113 - mp_next(mp_encode(1234567891234567890.0987654321987654321))
+ ok 114 - mp_decode(mp_encode(1234567891234567890.0987654321987654321) len
+ ok 115 - mp_decode(mp_encode(1234567891234567890.0987654321987654321)) value
+ ok 116 - mp_decode(mp_encode(1234567891234567890.0987654321987654321)) scale
+ ok 117 - str(mp_decode(mp_encode(1234567891234567890.0987654321987654321))) == 1234567891234567890.0987654321987654321
+ ok 118 - mp_ext_type is MP_DECIMAL
+ ok 119 - decimal_unpack() after mp_decode_extl()
+ ok 120 - decimal_unpack() after mp_decode_extl() value
+ ok 121 - decimal_unpack() after mp_decode_extl() len
+ ok 122 - mp_sizeof_decimal(-1234567891234567890.0987654321987654321)
+ ok 123 - mp_sizeof_decimal(-1234567891234567890.0987654321987654321) == len(mp_encode_decimal(-1234567891234567890.0987654321987654321))
+ ok 124 - mp_next(mp_encode(-1234567891234567890.0987654321987654321))
+ ok 125 - mp_decode(mp_encode(-1234567891234567890.0987654321987654321) len
+ ok 126 - mp_decode(mp_encode(-1234567891234567890.0987654321987654321)) value
+ ok 127 - mp_decode(mp_encode(-1234567891234567890.0987654321987654321)) scale
+ ok 128 - str(mp_decode(mp_encode(-1234567891234567890.0987654321987654321))) == -1234567891234567890.0987654321987654321
+ ok 129 - mp_ext_type is MP_DECIMAL
+ ok 130 - decimal_unpack() after mp_decode_extl()
+ ok 131 - decimal_unpack() after mp_decode_extl() value
+ ok 132 - decimal_unpack() after mp_decode_extl() len
+ ok 133 - mp_sizeof_decimal(0.0000000000000000000000000000000000001)
+ ok 134 - mp_sizeof_decimal(0.0000000000000000000000000000000000001) == len(mp_encode_decimal(0.0000000000000000000000000000000000001))
+ ok 135 - mp_next(mp_encode(0.0000000000000000000000000000000000001))
+ ok 136 - mp_decode(mp_encode(0.0000000000000000000000000000000000001) len
+ ok 137 - mp_decode(mp_encode(0.0000000000000000000000000000000000001)) value
+ ok 138 - mp_decode(mp_encode(0.0000000000000000000000000000000000001)) scale
+ ok 139 - str(mp_decode(mp_encode(0.0000000000000000000000000000000000001))) == 0.0000000000000000000000000000000000001
+ ok 140 - mp_ext_type is MP_DECIMAL
+ ok 141 - decimal_unpack() after mp_decode_extl()
+ ok 142 - decimal_unpack() after mp_decode_extl() value
+ ok 143 - decimal_unpack() after mp_decode_extl() len
+ ok 144 - mp_sizeof_decimal(-0.0000000000000000000000000000000000001)
+ ok 145 - mp_sizeof_decimal(-0.0000000000000000000000000000000000001) == len(mp_encode_decimal(-0.0000000000000000000000000000000000001))
+ ok 146 - mp_next(mp_encode(-0.0000000000000000000000000000000000001))
+ ok 147 - mp_decode(mp_encode(-0.0000000000000000000000000000000000001) len
+ ok 148 - mp_decode(mp_encode(-0.0000000000000000000000000000000000001)) value
+ ok 149 - mp_decode(mp_encode(-0.0000000000000000000000000000000000001)) scale
+ ok 150 - str(mp_decode(mp_encode(-0.0000000000000000000000000000000000001))) == -0.0000000000000000000000000000000000001
+ ok 151 - mp_ext_type is MP_DECIMAL
+ ok 152 - decimal_unpack() after mp_decode_extl()
+ ok 153 - decimal_unpack() after mp_decode_extl() value
+ ok 154 - decimal_unpack() after mp_decode_extl() len
+ ok 155 - mp_sizeof_decimal(0.00000000000000000000000000000000000001)
+ ok 156 - mp_sizeof_decimal(0.00000000000000000000000000000000000001) == len(mp_encode_decimal(0.00000000000000000000000000000000000001))
+ ok 157 - mp_next(mp_encode(0.00000000000000000000000000000000000001))
+ ok 158 - mp_decode(mp_encode(0.00000000000000000000000000000000000001) len
+ ok 159 - mp_decode(mp_encode(0.00000000000000000000000000000000000001)) value
+ ok 160 - mp_decode(mp_encode(0.00000000000000000000000000000000000001)) scale
+ ok 161 - str(mp_decode(mp_encode(0.00000000000000000000000000000000000001))) == 0.00000000000000000000000000000000000001
+ ok 162 - mp_ext_type is MP_DECIMAL
+ ok 163 - decimal_unpack() after mp_decode_extl()
+ ok 164 - decimal_unpack() after mp_decode_extl() value
+ ok 165 - decimal_unpack() after mp_decode_extl() len
+ ok 166 - mp_sizeof_decimal(-0.00000000000000000000000000000000000001)
+ ok 167 - mp_sizeof_decimal(-0.00000000000000000000000000000000000001) == len(mp_encode_decimal(-0.00000000000000000000000000000000000001))
+ ok 168 - mp_next(mp_encode(-0.00000000000000000000000000000000000001))
+ ok 169 - mp_decode(mp_encode(-0.00000000000000000000000000000000000001) len
+ ok 170 - mp_decode(mp_encode(-0.00000000000000000000000000000000000001)) value
+ ok 171 - mp_decode(mp_encode(-0.00000000000000000000000000000000000001)) scale
+ ok 172 - str(mp_decode(mp_encode(-0.00000000000000000000000000000000000001))) == -0.00000000000000000000000000000000000001
+ ok 173 - mp_ext_type is MP_DECIMAL
+ ok 174 - decimal_unpack() after mp_decode_extl()
+ ok 175 - decimal_unpack() after mp_decode_extl() value
+ ok 176 - decimal_unpack() after mp_decode_extl() len
+ ok 177 - mp_sizeof_decimal(99999999999999999999999999999999999999)
+ ok 178 - mp_sizeof_decimal(99999999999999999999999999999999999999) == len(mp_encode_decimal(99999999999999999999999999999999999999))
+ ok 179 - mp_next(mp_encode(99999999999999999999999999999999999999))
+ ok 180 - mp_decode(mp_encode(99999999999999999999999999999999999999) len
+ ok 181 - mp_decode(mp_encode(99999999999999999999999999999999999999)) value
+ ok 182 - mp_decode(mp_encode(99999999999999999999999999999999999999)) scale
+ ok 183 - str(mp_decode(mp_encode(99999999999999999999999999999999999999))) == 99999999999999999999999999999999999999
+ ok 184 - mp_ext_type is MP_DECIMAL
+ ok 185 - decimal_unpack() after mp_decode_extl()
+ ok 186 - decimal_unpack() after mp_decode_extl() value
+ ok 187 - decimal_unpack() after mp_decode_extl() len
+ ok 188 - mp_sizeof_decimal(-99999999999999999999999999999999999999)
+ ok 189 - mp_sizeof_decimal(-99999999999999999999999999999999999999) == len(mp_encode_decimal(-99999999999999999999999999999999999999))
+ ok 190 - mp_next(mp_encode(-99999999999999999999999999999999999999))
+ ok 191 - mp_decode(mp_encode(-99999999999999999999999999999999999999) len
+ ok 192 - mp_decode(mp_encode(-99999999999999999999999999999999999999)) value
+ ok 193 - mp_decode(mp_encode(-99999999999999999999999999999999999999)) scale
+ ok 194 - str(mp_decode(mp_encode(-99999999999999999999999999999999999999))) == -99999999999999999999999999999999999999
+ ok 195 - mp_ext_type is MP_DECIMAL
+ ok 196 - decimal_unpack() after mp_decode_extl()
+ ok 197 - decimal_unpack() after mp_decode_extl() value
+ ok 198 - decimal_unpack() after mp_decode_extl() len
+ok 280 - subtests
diff --git a/third_party/lua-cjson/lua_cjson.c b/third_party/lua-cjson/lua_cjson.c
index d8244054c..75e99512b 100644
--- a/third_party/lua-cjson/lua_cjson.c
+++ b/third_party/lua-cjson/lua_cjson.c
@@ -410,6 +410,11 @@ static void json_append_data(lua_State *l, struct luaL_serializer *cfg,
return json_append_nil(cfg, json); /* Limit nested arrays */
json_append_array(l, cfg, current_depth, json, field.size);
return;
+ case MP_FIELD_DECIMAL:
+ {
+ const char *str = decimal_to_string(field.decval);
+ return json_append_string(cfg, json, str, strlen(str));
+ }
case MP_FIELD_UNKNOWN:
/* handled by luaL_convertfield */
assert(false);
diff --git a/third_party/lua-yaml/lyaml.cc b/third_party/lua-yaml/lyaml.cc
index 23d10335b..9f6f90d24 100644
--- a/third_party/lua-yaml/lyaml.cc
+++ b/third_party/lua-yaml/lyaml.cc
@@ -49,6 +49,7 @@ extern "C" {
#include "b64.h"
} /* extern "C" */
#include "lua/utils.h"
+#include "lib/core/decimal.h"
#define LUAYAML_TAG_PREFIX "tag:yaml.org,2002:"
@@ -692,6 +693,10 @@ static int dump_node(struct lua_yaml_dumper *dumper)
str = "null";
len = 4;
break;
+ case MP_FIELD_DECIMAL:
+ str = decimal_to_string(field.decval);
+ len = strlen(str);
+ break;
case MP_FIELD_UNKNOWN:
assert(0); /* checked by luaL_checkfield() */
break;
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack
2019-08-08 11:55 ` [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack Serge Petrenko
@ 2019-08-12 21:29 ` Konstantin Osipov
2019-08-12 21:34 ` Konstantin Osipov
1 sibling, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:29 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
> This patch adds the methods necessary to encode and decode decimals to
> MsgPack. MsgPack EXT type (MP_EXT) together with a new extension type
> MP_DECIMAL is used as a record header.
>
> The decimal MsgPack representation looks like this:
> +--------+-------------------+------------+===============+
> | MP_EXT | length (optional) | MP_DECIMAL | PackedDecimal |
> +--------+-------------------+------------+===============+
> The whole record may be encoded and decoded with
> mp_encode_decimal() and mp_decode_decimal(). This is equivalent to
> performing mp_encode_extl()/mp_decode_extl() on the first 3 fields and
> decimal_pack/unpack() on the PackedDecimal field.
>
> It is also possible to decode and encode decimals to msgpack from lua,
> which means you can insert decimals into spaces, but only into unindexed
> fields for now.
I like that this commit consolidates mp encode/decode and decimal
in a separate file, but I don't see it remove dependency on
messagepack from decimal.h|c
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack
2019-08-08 11:55 ` [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack Serge Petrenko
2019-08-12 21:29 ` Konstantin Osipov
@ 2019-08-12 21:34 ` Konstantin Osipov
2019-08-13 14:01 ` Serge Petrenko
1 sibling, 1 reply; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:34 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
> index d84748b71..89ea26b41 100644
> --- a/src/lib/core/mp_user_types.h
> +++ b/src/lib/core/mp_user_types.h
> @@ -31,6 +31,12 @@
> * SUCH DAMAGE.
> */
>
> +#include "msgpuck.h"
why this include?
What is this header file standing (mp_user_tpyes.h) for?
User types are enum field_type. What other user types do you have
in mind? Perhaps you mean user-defined types? Well, decimal is not
a user defined type, it's built-in. The fact that it has to use
mp_ext format type in msgpack is just a coincidence of Tarantool
using msgpack for its internal data representation.
> +++ b/src/lib/core/mpstream.h
> @@ -32,6 +32,7 @@
> */
>
> #include "diag.h"
> +#include "decimal.h"
>
why this include?
Please, please, C is not PHP, you don't include everything around
just because it was a bad day.
> index 6fc2b8278..2e6b5c163 100644
> --- a/src/lua/msgpack.c
> +++ b/src/lua/msgpack.c
> @@ -41,6 +41,11 @@
> #include <small/region.h>
> #include <small/ibuf.h>
>
> +#include "lua/decimal.h"
> +#include "lib/core/decimal.h"
> +#include "lib/core/mp_decimal.h"
> +#include "lib/core/mp_user_types.h"
> +
This is a dependency hell.
Why did you add a separate header for decimal encoding/encoding if
you have to include *everything* in here anyway? Noone will ever
be able to understand why you decided to include all of these
here.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack
2019-08-12 21:34 ` Konstantin Osipov
@ 2019-08-13 14:01 ` Serge Petrenko
2019-08-14 22:25 ` Konstantin Osipov
0 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-13 14:01 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: Vladimir Davydov, tarantool-patches
> 13 авг. 2019 г., в 0:34, Konstantin Osipov <kostja@tarantool.org> написал(а):
>
> * Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
>> index d84748b71..89ea26b41 100644
>> --- a/src/lib/core/mp_user_types.h
>> +++ b/src/lib/core/mp_user_types.h
>> @@ -31,6 +31,12 @@
>> * SUCH DAMAGE.
>> */
>>
>> +#include "msgpuck.h"
>
> why this include?
For mp_type in msgpack_to_field_type() and field_type_to_msgpack().
>
>
> What is this header file standing (mp_user_tpyes.h) for?
>
> User types are enum field_type. What other user types do you have
> in mind? Perhaps you mean user-defined types? Well, decimal is not
> a user defined type, it's built-in. The fact that it has to use
> mp_ext format type in msgpack is just a coincidence of Tarantool
> using msgpack for its internal data representation.
Ok, we may name the file extension_types or ext_types or mp_ext_types.
>
>> +++ b/src/lib/core/mpstream.h
>> @@ -32,6 +32,7 @@
>> */
>>
>> #include "diag.h"
>> +#include "decimal.h"
>>
>
> why this include?
For decimal_t. Because
decimal_t is a typedef for decNumber, and decNumber is a typedef
for an anonymous struct.
I can patch decNumber so that we have a struct decNumber. Then we
will be able to get away with typedef struct decNumber decimal_t in
such headers.
>
> Please, please, C is not PHP, you don't include everything around
> just because it was a bad day.
>
>> index 6fc2b8278..2e6b5c163 100644
>> --- a/src/lua/msgpack.c
>> +++ b/src/lua/msgpack.c
>> @@ -41,6 +41,11 @@
>> #include <small/region.h>
>> #include <small/ibuf.h>
>>
>> +#include "lua/decimal.h"
>> +#include "lib/core/decimal.h"
>> +#include "lib/core/mp_decimal.h"
>> +#include "lib/core/mp_user_types.h"
>> +
>
> This is a dependency hell.
>
> Why did you add a separate header for decimal encoding/encoding if
> you have to include *everything* in here anyway? Noone will ever
> be able to understand why you decided to include all of these
> here.
Do you want me to add comments describing why we need each header?
>
>
>
> --
> Konstantin Osipov, Moscow, Russia
--
Serge Petrenko
sergepetrenko@tarantool.org
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack
2019-08-13 14:01 ` Serge Petrenko
@ 2019-08-14 22:25 ` Konstantin Osipov
0 siblings, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-14 22:25 UTC (permalink / raw)
To: Serge Petrenko; +Cc: Vladimir Davydov, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/13 17:05]:
> > This is a dependency hell.
> >
> > Why did you add a separate header for decimal encoding/encoding if
> > you have to include *everything* in here anyway? Noone will ever
> > be able to understand why you decided to include all of these
> > here.
>
> Do you want me to add comments describing why we need each header?
My point is, including mp_decimal.h should be enough.
>
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 7/8] decimal: add conversions to (u)int64_t
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
` (5 preceding siblings ...)
2019-08-08 11:55 ` [PATCH v2 6/8] decimal: allow to encode/decode decimals as MsgPack Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-12 21:39 ` Konstantin Osipov
2019-08-08 11:55 ` [PATCH v2 8/8] decimal: allow to index decimals Serge Petrenko
7 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
Update decNumber library, add methods to convert decimals to uint64_t
and int64_t, add unit tests.
Also replace decimal_round() function with decimal_round_with_mode() to
allow setting rounding mode.
We need to round with mode DEC_ROUND_DOWN in to_int64 conversions in
order to be consistent with double to int conversions.
It will be needed to compute hints for decimal fields.
Prerequisite #4333
---
src/lib/core/decimal.c | 57 +++++++++++++++++++++++++++++--
src/lib/core/decimal.h | 19 +++++++++++
test/unit/decimal.c | 66 ++++++++++++++++++++++++++++++++++-
test/unit/decimal.result | 74 ++++++++++++++++++++++++++++++++++++++--
third_party/decNumber | 2 +-
5 files changed, 210 insertions(+), 8 deletions(-)
diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
index 840aa5dfe..19b5096ed 100644
--- a/src/lib/core/decimal.c
+++ b/src/lib/core/decimal.c
@@ -157,6 +157,45 @@ decimal_to_string(const decimal_t *dec)
return buf;
}
+static decimal_t *
+decimal_to_integer(decimal_t *dec)
+{
+ decimal_t z;
+ decNumberZero(&z);
+ if (decimal_scale(dec) != 0) {
+ /*
+ * Rounding mode is important here.
+ * We want to be consistent with double
+ * to int conversion so that comparison
+ * hints work correctly.
+ */
+ decimal_floor(dec, 0);
+ }
+ /* Zero the number exponent for decNumberToInt64. */
+ decNumberRescale(dec, dec, &z, &decimal_context);
+ return decimal_check_status(dec, &decimal_context);
+}
+
+decimal_t *
+decimal_to_int64(const decimal_t *dec, int64_t *num)
+{
+ decimal_t d = *dec;
+ if (decimal_to_integer(&d) == NULL)
+ return NULL;
+ *num = decNumberToInt64(&d, &decimal_context);
+ return decimal_check_status(&d, &decimal_context);
+}
+
+decimal_t *
+decimal_to_uint64(const decimal_t *dec, uint64_t *num)
+{
+ decimal_t d = *dec;
+ if (decimal_to_integer(&d) == NULL)
+ return NULL;
+ *num = decNumberToUInt64(&d, &decimal_context);
+ return decimal_check_status(&d, &decimal_context);
+}
+
int
decimal_compare(const decimal_t *lhs, const decimal_t *rhs)
{
@@ -167,8 +206,8 @@ decimal_compare(const decimal_t *lhs, const decimal_t *rhs)
return r;
}
-decimal_t *
-decimal_round(decimal_t *dec, int scale)
+static decimal_t *
+decimal_round_with_mode(decimal_t *dec, int scale, enum rounding mode)
{
if (scale < 0 || scale > DECIMAL_MAX_DIGITS)
return NULL;
@@ -181,7 +220,7 @@ decimal_round(decimal_t *dec, int scale)
ndig, /* Precision */
ndig, /* emax */
scale != 0 ? -1 : 0, /* emin */
- DECIMAL_ROUNDING, /* rounding */
+ mode, /* rounding */
0, /* no traps */
0, /* zero status */
0 /* no clamping */
@@ -192,6 +231,18 @@ decimal_round(decimal_t *dec, int scale)
return dec;
}
+inline decimal_t *
+decimal_round(decimal_t *dec, int scale)
+{
+ return decimal_round_with_mode(dec, scale, DECIMAL_ROUNDING);
+}
+
+inline decimal_t *
+decimal_floor(decimal_t *dec, int scale)
+{
+ return decimal_round_with_mode(dec, scale, DEC_ROUND_DOWN);
+}
+
decimal_t *
decimal_trim(decimal_t *dec)
{
diff --git a/src/lib/core/decimal.h b/src/lib/core/decimal.h
index 6e2cd3ce7..cf41a5052 100644
--- a/src/lib/core/decimal.h
+++ b/src/lib/core/decimal.h
@@ -101,6 +101,18 @@ decimal_from_uint64(decimal_t *dec, uint64_t num);
const char *
decimal_to_string(const decimal_t *dec);
+/**
+ * Convert a given decimal to int64_t
+ * \param[out] num - the result
+ * @return NULL if \a dec doesn't fit into int64_t
+ */
+decimal_t *
+decimal_to_int64(const decimal_t *dec, int64_t *num);
+
+/** \sa decimal_to_int64 */
+decimal_t *
+decimal_to_uint64(const decimal_t *dec, uint64_t *num);
+
/**
* Compare 2 decimal values.
* @return -1, lhs < rhs,
@@ -122,6 +134,13 @@ decimal_compare(const decimal_t *lhs, const decimal_t *rhs);
decimal_t *
decimal_round(decimal_t *dec, int scale);
+/**
+ * Round a decimal towards zero.
+ * \sa decimal_round
+ */
+decimal_t *
+decimal_floor(decimal_t *dec, int scale);
+
/**
* Remove trailing zeros from the fractional part of a number.
* @return \a dec with trimmed fractional zeros.
diff --git a/test/unit/decimal.c b/test/unit/decimal.c
index a70fe9a9e..9afe17432 100644
--- a/test/unit/decimal.c
+++ b/test/unit/decimal.c
@@ -5,6 +5,7 @@
#include "msgpuck.h"
#include <limits.h>
#include <string.h>
+#include <inttypes.h>
#include <float.h> /* DBL_DIG */
#define success(x) x
@@ -117,6 +118,15 @@ char buf[32];
is(strcmp(decimal_to_string(&d2), str), 0, "str(decimal_unpack(decimal_pack("str")) == "str);\
})
+#define test_toint(type, num, out_fmt) ({\
+ decimal_t dec;\
+ type##_t val;\
+ decimal_from_##type(&dec, num);\
+ isnt(decimal_to_##type(&dec, &val), NULL, "Conversion of %"out_fmt\
+ " to decimal and back to "#type" successful", (type##_t) num);\
+ is(val, num, "Conversion back to "#type" correct");\
+})
+
static int
test_pack_unpack(void)
{
@@ -192,10 +202,62 @@ test_mp_decimal(void)
return check_plan();
}
+static int
+test_to_int(void)
+{
+ plan(66);
+
+ test_toint(uint64, ULLONG_MAX, PRIu64);
+ test_toint(int64, LLONG_MAX, PRId64);
+ test_toint(int64, LLONG_MIN, PRId64);
+ test_toint(uint64, 0, PRIu64);
+ test_toint(int64, 0, PRId64);
+ test_toint(int64, -1, PRId64);
+
+ /* test some arbitrary values. */
+ test_toint(uint64, ULLONG_MAX / 157, PRIu64);
+ test_toint(int64, LLONG_MAX / 157, PRId64);
+ test_toint(int64, LLONG_MIN / 157, PRId64);
+
+ test_toint(uint64, ULLONG_MAX / 157 / 151, PRIu64);
+ test_toint(int64, LLONG_MAX / 157 / 151, PRId64);
+ test_toint(int64, LLONG_MIN / 157 / 151, PRId64);
+
+ test_toint(uint64, ULLONG_MAX / 157 / 151 / 149, PRIu64);
+ test_toint(int64, LLONG_MAX / 157 / 151 / 149, PRId64);
+ test_toint(int64, LLONG_MIN / 157 / 151 / 149, PRId64);
+
+ test_toint(uint64, ULLONG_MAX / 157 / 151 / 149 / 139, PRIu64);
+ test_toint(int64, LLONG_MAX / 157 / 151 / 149 / 139, PRId64);
+ test_toint(int64, LLONG_MIN / 157 / 151 / 149 / 139, PRId64);
+
+ test_toint(uint64, ULLONG_MAX / 157 / 151 / 149 / 139 / 137, PRIu64);
+ test_toint(int64, LLONG_MAX / 156 / 151 / 149 / 139 / 137, PRId64);
+ test_toint(int64, LLONG_MIN / 156 / 151 / 149 / 139 / 137, PRId64);
+
+ test_toint(uint64, UINT_MAX, PRIu64);
+ test_toint(int64, INT_MAX, PRId64);
+ test_toint(int64, INT_MIN, PRId64);
+
+ test_toint(uint64, UINT_MAX / 157, PRIu64); /* ~ 27356479 */
+ test_toint(int64, INT_MAX / 157, PRId64);
+ test_toint(int64, INT_MIN / 157, PRId64);
+
+ test_toint(uint64, UINT_MAX / 157 / 151, PRIu64); /* ~ 181168 */
+ test_toint(int64, INT_MAX / 157 / 151, PRId64);
+ test_toint(int64, INT_MIN / 157 / 151, PRId64);
+
+ test_toint(uint64, UINT_MAX / 157 / 151 / 149, PRIu64); /* ~ 1215 */
+ test_toint(int64, INT_MAX / 157 / 151 / 149, PRId64);
+ test_toint(int64, INT_MIN / 157 / 151 / 149, PRId64);
+
+ return check_plan();
+}
+
int
main(void)
{
- plan(280);
+ plan(281);
dectest(314, 271, uint64, uint64_t);
dectest(65535, 23456, uint64, uint64_t);
@@ -255,6 +317,8 @@ main(void)
dectest_op1_fail(log10, -1);
dectest_op1_fail(sqrt, -10);
+ test_to_int();
+
test_pack_unpack();
test_mp_decimal();
diff --git a/test/unit/decimal.result b/test/unit/decimal.result
index e6812a8dd..e8432fb36 100644
--- a/test/unit/decimal.result
+++ b/test/unit/decimal.result
@@ -1,4 +1,4 @@
-1..280
+1..281
ok 1 - decimal(314)
ok 2 - decimal(271)
ok 3 - decimal(314) + decimal(271)
@@ -277,6 +277,74 @@ ok 275 - decimal_from_string(-1)
ok 276 - decimal_log10(-1) - error on wrong operands.
ok 277 - decimal_from_string(-10)
ok 278 - decimal_sqrt(-10) - error on wrong operands.
+ 1..66
+ ok 1 - Conversion of 18446744073709551615 to decimal and back to uint64 successful
+ ok 2 - Conversion back to uint64 correct
+ ok 3 - Conversion of 9223372036854775807 to decimal and back to int64 successful
+ ok 4 - Conversion back to int64 correct
+ ok 5 - Conversion of -9223372036854775808 to decimal and back to int64 successful
+ ok 6 - Conversion back to int64 correct
+ ok 7 - Conversion of 0 to decimal and back to uint64 successful
+ ok 8 - Conversion back to uint64 correct
+ ok 9 - Conversion of 0 to decimal and back to int64 successful
+ ok 10 - Conversion back to int64 correct
+ ok 11 - Conversion of -1 to decimal and back to int64 successful
+ ok 12 - Conversion back to int64 correct
+ ok 13 - Conversion of 117495185182863386 to decimal and back to uint64 successful
+ ok 14 - Conversion back to uint64 correct
+ ok 15 - Conversion of 58747592591431693 to decimal and back to int64 successful
+ ok 16 - Conversion back to int64 correct
+ ok 17 - Conversion of -58747592591431693 to decimal and back to int64 successful
+ ok 18 - Conversion back to int64 correct
+ ok 19 - Conversion of 778113809158035 to decimal and back to uint64 successful
+ ok 20 - Conversion back to uint64 correct
+ ok 21 - Conversion of 389056904579017 to decimal and back to int64 successful
+ ok 22 - Conversion back to int64 correct
+ ok 23 - Conversion of -389056904579017 to decimal and back to int64 successful
+ ok 24 - Conversion back to int64 correct
+ ok 25 - Conversion of 5222240329919 to decimal and back to uint64 successful
+ ok 26 - Conversion back to uint64 correct
+ ok 27 - Conversion of 2611120164959 to decimal and back to int64 successful
+ ok 28 - Conversion back to int64 correct
+ ok 29 - Conversion of -2611120164959 to decimal and back to int64 successful
+ ok 30 - Conversion back to int64 correct
+ ok 31 - Conversion of 37570074315 to decimal and back to uint64 successful
+ ok 32 - Conversion back to uint64 correct
+ ok 33 - Conversion of 18785037157 to decimal and back to int64 successful
+ ok 34 - Conversion back to int64 correct
+ ok 35 - Conversion of -18785037157 to decimal and back to int64 successful
+ ok 36 - Conversion back to int64 correct
+ ok 37 - Conversion of 274234119 to decimal and back to uint64 successful
+ ok 38 - Conversion back to uint64 correct
+ ok 39 - Conversion of 137996015 to decimal and back to int64 successful
+ ok 40 - Conversion back to int64 correct
+ ok 41 - Conversion of -137996015 to decimal and back to int64 successful
+ ok 42 - Conversion back to int64 correct
+ ok 43 - Conversion of 4294967295 to decimal and back to uint64 successful
+ ok 44 - Conversion back to uint64 correct
+ ok 45 - Conversion of 2147483647 to decimal and back to int64 successful
+ ok 46 - Conversion back to int64 correct
+ ok 47 - Conversion of -2147483648 to decimal and back to int64 successful
+ ok 48 - Conversion back to int64 correct
+ ok 49 - Conversion of 27356479 to decimal and back to uint64 successful
+ ok 50 - Conversion back to uint64 correct
+ ok 51 - Conversion of 13678239 to decimal and back to int64 successful
+ ok 52 - Conversion back to int64 correct
+ ok 53 - Conversion of -13678239 to decimal and back to int64 successful
+ ok 54 - Conversion back to int64 correct
+ ok 55 - Conversion of 181168 to decimal and back to uint64 successful
+ ok 56 - Conversion back to uint64 correct
+ ok 57 - Conversion of 90584 to decimal and back to int64 successful
+ ok 58 - Conversion back to int64 correct
+ ok 59 - Conversion of -90584 to decimal and back to int64 successful
+ ok 60 - Conversion back to int64 correct
+ ok 61 - Conversion of 1215 to decimal and back to uint64 successful
+ ok 62 - Conversion back to uint64 correct
+ ok 63 - Conversion of 607 to decimal and back to int64 successful
+ ok 64 - Conversion back to int64 correct
+ ok 65 - Conversion of -607 to decimal and back to int64 successful
+ ok 66 - Conversion back to int64 correct
+ok 279 - subtests
1..151
ok 1 - decimal_len(0)
ok 2 - decimal_len(0) == len(decimal_pack(0)
@@ -429,7 +497,7 @@ ok 278 - decimal_sqrt(-10) - error on wrong operands.
ok 149 - positive exponent number is packed/unpacked correctly
ok 150 - unpack malformed decimal fails
ok 151 - decode malformed decimal preserves buffer position
-ok 279 - subtests
+ok 280 - subtests
1..198
ok 1 - mp_sizeof_decimal(0)
ok 2 - mp_sizeof_decimal(0) == len(mp_encode_decimal(0))
@@ -629,4 +697,4 @@ ok 279 - subtests
ok 196 - decimal_unpack() after mp_decode_extl()
ok 197 - decimal_unpack() after mp_decode_extl() value
ok 198 - decimal_unpack() after mp_decode_extl() len
-ok 280 - subtests
+ok 281 - subtests
diff --git a/third_party/decNumber b/third_party/decNumber
index ee540fca6..878ed752f 160000
--- a/third_party/decNumber
+++ b/third_party/decNumber
@@ -1 +1 @@
-Subproject commit ee540fca6a44b3a2df7258dc7a1ec612cbf84dce
+Subproject commit 878ed752f2453cd5e73587e67f7782aec9181a22
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 7/8] decimal: add conversions to (u)int64_t
2019-08-08 11:55 ` [PATCH v2 7/8] decimal: add conversions to (u)int64_t Serge Petrenko
@ 2019-08-12 21:39 ` Konstantin Osipov
2019-08-13 14:18 ` Serge Petrenko
0 siblings, 1 reply; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:39 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
> Update decNumber library, add methods to convert decimals to uint64_t
> and int64_t, add unit tests.
> Also replace decimal_round() function with decimal_round_with_mode() to
> allow setting rounding mode.
> We need to round with mode DEC_ROUND_DOWN in to_int64 conversions in
> order to be consistent with double to int conversions.
> It will be needed to compute hints for decimal fields.
>
> Prerequisite #4333
> ---
> src/lib/core/decimal.c | 57 +++++++++++++++++++++++++++++--
> src/lib/core/decimal.h | 19 +++++++++++
> test/unit/decimal.c | 66 ++++++++++++++++++++++++++++++++++-
> test/unit/decimal.result | 74 ++++++++++++++++++++++++++++++++++++++--
> third_party/decNumber | 2 +-
> 5 files changed, 210 insertions(+), 8 deletions(-)
>
> diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
> index 840aa5dfe..19b5096ed 100644
> --- a/src/lib/core/decimal.c
> +++ b/src/lib/core/decimal.c
> @@ -157,6 +157,45 @@ decimal_to_string(const decimal_t *dec)
> return buf;
> }
>
> +static decimal_t *
> +decimal_to_integer(decimal_t *dec)
Why do you think it's the best hint option? Why not keep 2-5 decimal
digits in a hint, as most currencies use? Would it slow down the
hing making process? You call decimalRescale anyway, it seems.
Is decimalRescale optimized to be a no-op if there is no need to
rescale?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 7/8] decimal: add conversions to (u)int64_t
2019-08-12 21:39 ` Konstantin Osipov
@ 2019-08-13 14:18 ` Serge Petrenko
2019-08-14 22:26 ` Konstantin Osipov
0 siblings, 1 reply; 35+ messages in thread
From: Serge Petrenko @ 2019-08-13 14:18 UTC (permalink / raw)
To: Konstantin Osipov; +Cc: Vladimir Davydov, tarantool-patches
[-- Attachment #1: Type: text/plain, Size: 2082 bytes --]
> 13 авг. 2019 г., в 0:39, Konstantin Osipov <kostja@tarantool.org> написал(а):
>
> * Serge Petrenko <sergepetrenko@tarantool.org <mailto:sergepetrenko@tarantool.org>> [19/08/08 15:01]:
>> Update decNumber library, add methods to convert decimals to uint64_t
>> and int64_t, add unit tests.
>> Also replace decimal_round() function with decimal_round_with_mode() to
>> allow setting rounding mode.
>> We need to round with mode DEC_ROUND_DOWN in to_int64 conversions in
>> order to be consistent with double to int conversions.
>> It will be needed to compute hints for decimal fields.
>>
>> Prerequisite #4333
>> ---
>> src/lib/core/decimal.c | 57 +++++++++++++++++++++++++++++--
>> src/lib/core/decimal.h | 19 +++++++++++
>> test/unit/decimal.c | 66 ++++++++++++++++++++++++++++++++++-
>> test/unit/decimal.result | 74 ++++++++++++++++++++++++++++++++++++++--
>> third_party/decNumber | 2 +-
>> 5 files changed, 210 insertions(+), 8 deletions(-)
>>
>> diff --git a/src/lib/core/decimal.c b/src/lib/core/decimal.c
>> index 840aa5dfe..19b5096ed 100644
>> --- a/src/lib/core/decimal.c
>> +++ b/src/lib/core/decimal.c
>> @@ -157,6 +157,45 @@ decimal_to_string(const decimal_t *dec)
>> return buf;
>> }
>>
>> +static decimal_t *
>> +decimal_to_integer(decimal_t *dec)
>
> Why do you think it's the best hint option? Why not keep 2-5 decimal
> digits in a hint, as most currencies use? Would it slow down the
> hing making process? You call decimalRescale anyway, it seems.
It wouldn’t slow down hint calculation (timing difference is tiny).
It will slow down decimal comparison with doubles and integers, though.
We will get faster comparison for decimals that differ in first 2-5 fractional
digits at the cost of having to cast every double and int to decimal for comparison
with decimals.
Is this worth it?
>
> Is decimalRescale optimized to be a no-op if there is no need to
> rescale?
Yes
>
> --
> Konstantin Osipov, Moscow, Russia
--
Serge Petrenko
sergepetrenko@tarantool.org
[-- Attachment #2: Type: text/html, Size: 10953 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 7/8] decimal: add conversions to (u)int64_t
2019-08-13 14:18 ` Serge Petrenko
@ 2019-08-14 22:26 ` Konstantin Osipov
2019-08-14 22:29 ` Konstantin Osipov
0 siblings, 1 reply; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-14 22:26 UTC (permalink / raw)
To: Serge Petrenko; +Cc: Vladimir Davydov, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/13 17:20]:
> > Why do you think it's the best hint option? Why not keep 2-5 decimal
> > digits in a hint, as most currencies use? Would it slow down the
> > hing making process? You call decimalRescale anyway, it seems.
>
> It wouldn’t slow down hint calculation (timing difference is tiny).
> It will slow down decimal comparison with doubles and integers, though.
>
> We will get faster comparison for decimals that differ in first 2-5 fractional
> digits at the cost of having to cast every double and int to decimal for comparison
> with decimals.
> Is this worth it?
Please ask yourself this. How often do you think you will compare
a decimal with another decimal, compared to a decimal with a
float?
> > Is decimalRescale optimized to be a no-op if there is no need to
> > rescale?
>
> Yes
Thanks.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v2 7/8] decimal: add conversions to (u)int64_t
2019-08-14 22:26 ` Konstantin Osipov
@ 2019-08-14 22:29 ` Konstantin Osipov
0 siblings, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-14 22:29 UTC (permalink / raw)
To: Serge Petrenko; +Cc: Vladimir Davydov, tarantool-patches
* Konstantin Osipov <kostja@tarantool.org> [19/08/15 01:26]:
> > > Why do you think it's the best hint option? Why not keep 2-5 decimal
> > > digits in a hint, as most currencies use? Would it slow down the
> > > hing making process? You call decimalRescale anyway, it seems.
> >
> > It wouldn’t slow down hint calculation (timing difference is tiny).
> > It will slow down decimal comparison with doubles and integers, though.
> >
> > We will get faster comparison for decimals that differ in first 2-5 fractional
> > digits at the cost of having to cast every double and int to decimal for comparison
> > with decimals.
> > Is this worth it?
>
> Please ask yourself this. How often do you think you will compare
> a decimal with another decimal, compared to a decimal with a
> float?
On the other hand, how often do you actually have distinct
fractional parts in a decimal... Decimal is mostly used to store
money/currency and most of the time fractional part is zero
nowadays, no matter what the currency is.
So I guess your approach is OK.
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v2 8/8] decimal: allow to index decimals
2019-08-08 11:55 [PATCH v2 0/8] Decimal indices Serge Petrenko
` (6 preceding siblings ...)
2019-08-08 11:55 ` [PATCH v2 7/8] decimal: add conversions to (u)int64_t Serge Petrenko
@ 2019-08-08 11:55 ` Serge Petrenko
2019-08-08 13:42 ` Serge Petrenko
2019-08-12 21:41 ` Konstantin Osipov
7 siblings, 2 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 11:55 UTC (permalink / raw)
To: vdavydov.dev; +Cc: tarantool-patches, kostja, Serge Petrenko
Closes #4333
@TarantoolBot document
Title: Document decimal field type.
Decimals may now be stored in spaces. A corresponding field type is
introduced: 'decimal'. Decimal values are also allowed in 'scalar',
'any' and 'number' fields.
'decimal' field type is appropriate for both memtx HASH and TREE
indices, as well as for vinyl TREE index.
```
To create an index 'pk' over a decimal field, say
```
tarantool> box.space.test:create_index('pk', {parts={1, 'decimal'}})
---
- unique: true
parts:
- type: decimal
is_nullable: false
fieldno: 1
id: 0
space_id: 512
type: TREE
name: pk
...
```
Now you can insert some decimal values:
```
tarantool> for i = 1,10 do
> box.space.test:insert{decimal.new((i-5)/10)}
> end
---
...
```
tarantool> box.space.test:select{}
---
- - [-0.4]
- [-0.3]
- [-0.2]
- [-0.1]
- [0]
- [0.1]
- [0.2]
- [0.3]
- [0.4]
- [0.5]
...
```
Decimals may alse be inserted into `scalar` and `number` fields. In this
case all the number values are sorted correctly:
```
tarantool> box.schema.space.create('test')
tarantool> box.space.test:create_index('pk', {parts={1, 'number'}})
tarantool> box.space.test:insert{-1.0001, 'number'}
---
- [-1.0001, 'number']
...
tarantool> box.space.test:insert{decimal.new(-1.00001), 'decimal'}
---
- [-1.00001, 'decimal']
...
tarantool> box.space.test:insert{-1, 'number'}
---
- [-1, 'number']
...
tarantool> box.space.test:insert{decimal.new(-0.999), 'decimal'}
---
- [-0.999, 'decimal']
...
tarantool> box.space.test:insert{-0.998, 'number'}
---
- [-0.998, 'number']
...
tarantool> box.space.test:insert{-0.9, 'number'}
---
- [-0.9, 'number']
...
tarantool> box.space.test:insert{-0.95, 'number'}
---
- [-0.95, 'number']
...
tarantool> box.space.test:insert{decimal.new(-0.92), 'decimal'}
---
- [-0.92, 'decimal']
...
tarantool> box.space.test:insert{decimal.new(-0.971), 'decimal'}
---
- [-0.971, 'decimal']
...
tarantool> box.space.test:select{}
---
- - [-1.0001, 'number']
- [-1.00001, 'decimal']
- [-1, 'number']
- [-0.999, 'decimal']
- [-0.998, 'number']
- [-0.971, 'decimal']
- [-0.95, 'number']
- [-0.92, 'decimal']
- [-0.9, 'number']
...
```
Uniqueness is also preserved between decimals and other number types:
```
tarantool> box.space.test:insert{-0.92}
---
- error: Duplicate key exists in unique index 'pk' in space 'test'
...
tarantool> box.space.test:insert{decimal.new(-0.9)}
---
- error: Duplicate key exists in unique index 'pk' in space 'test'
...
```
You can also set decimal fields in space format:
```
tarantool> _ = box.schema.space.create('test')
---
...
tarantool> _ = box.space.test:create_index('pk')
---
...
tarantool> box.space.test:format{{name='id', type='unsigned'}, {name='balance', type='decimal'}}
---
...
tarantool> box.space.test:insert{1}
---
- error: Tuple field 2 required by space format is missing
...
tarantool> box.space.test:insert{1, 'string'}
---
- error: 'Tuple field 2 type does not match one required by operation: expected decimal'
...
tarantool> box.space.test:insert{1, 1.2345}
---
- error: 'Tuple field 2 type does not match one required by operation: expected decimal'
...
tarantool> box.space.test:insert{1, decimal.new('1337.420')}
---
- [1, 1337.420]
...
```
---
src/box/field_def.c | 30 ++--
src/box/field_def.h | 1 +
src/box/tuple_compare.cc | 104 ++++++++++++-
src/lib/core/decimal.h | 8 +
src/lib/core/mp_decimal.h | 8 +
test/engine/ddl.result | 85 ++++++++++-
test/engine/ddl.test.lua | 40 ++++-
test/engine/decimal.result | 275 +++++++++++++++++++++++++++++++++++
test/engine/decimal.test.lua | 83 +++++++++++
9 files changed, 617 insertions(+), 17 deletions(-)
diff --git a/src/box/field_def.c b/src/box/field_def.c
index 2fad81d42..b1ecb1f19 100644
--- a/src/box/field_def.c
+++ b/src/box/field_def.c
@@ -32,6 +32,7 @@
#include "field_def.h"
#include "trivia/util.h"
#include "key_def.h"
+#include "lua/utils.h"
const char *mp_type_strs[] = {
/* .MP_NIL = */ "nil",
@@ -52,13 +53,14 @@ const uint32_t field_mp_type[] = {
/* [FIELD_TYPE_UNSIGNED] = */ 1U << MP_FIELD_UINT,
/* [FIELD_TYPE_STRING] = */ 1U << MP_FIELD_STR,
/* [FIELD_TYPE_NUMBER] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT) |
- (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE),
+ (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE) | (1U << MP_FIELD_DECIMAL),
/* [FIELD_TYPE_INTEGER] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT),
/* [FIELD_TYPE_BOOLEAN] = */ 1U << MP_FIELD_BOOL,
/* [FIELD_TYPE_VARBINARY] = */ 1U << MP_FIELD_BIN,
/* [FIELD_TYPE_SCALAR] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT) |
(1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE) | (1U << MP_FIELD_STR) |
- (1U << MP_FIELD_BIN) | (1U << MP_FIELD_BOOL),
+ (1U << MP_FIELD_BIN) | (1U << MP_FIELD_BOOL) | (1U << MP_FIELD_DECIMAL),
+ /* [FIELD_TYPE_DECIMAL] = */ 1U << MP_FIELD_DECIMAL,
/* [FIELD_TYPE_ARRAY] = */ 1U << MP_FIELD_ARRAY,
/* [FIELD_TYPE_MAP] = */ (1U << MP_FIELD_MAP),
};
@@ -72,6 +74,7 @@ const char *field_type_strs[] = {
/* [FIELD_TYPE_BOOLEAN] = */ "boolean",
/* [FIELD_TYPE_VARBINARY] = */"varbinary",
/* [FIELD_TYPE_SCALAR] = */ "scalar",
+ /* [FIELD_TYPE_DECIMAL] = */ "decimal",
/* [FIELD_TYPE_ARRAY] = */ "array",
/* [FIELD_TYPE_MAP] = */ "map",
};
@@ -98,17 +101,18 @@ field_type_by_name_wrapper(const char *str, uint32_t len)
* values can be stored in the j type.
*/
static const bool field_type_compatibility[] = {
- /* ANY UNSIGNED STRING NUMBER INTEGER BOOLEAN VARBINARY SCALAR ARRAY MAP */
-/* ANY */ true, false, false, false, false, false, false, false, false, false,
-/* UNSIGNED */ true, true, false, true, true, false, false, true, false, false,
-/* STRING */ true, false, true, false, false, false, false, true, false, false,
-/* NUMBER */ true, false, false, true, false, false, false, true, false, false,
-/* INTEGER */ true, false, false, true, true, false, false, true, false, false,
-/* BOOLEAN */ true, false, false, false, false, true, false, true, false, false,
-/* VARBINARY*/ true, false, false, false, false, false, true, true, false, false,
-/* SCALAR */ true, false, false, false, false, false, false, true, false, false,
-/* ARRAY */ true, false, false, false, false, false, false, false, true, false,
-/* MAP */ true, false, false, false, false, false, false, false, false, true,
+ /* ANY UNSIGNED STRING NUMBER INTEGER BOOLEAN VARBINARY SCALAR DECIMAL ARRAY MAP */
+/* ANY */ true, false, false, false, false, false, false, false, false, false, false,
+/* UNSIGNED */ true, true, false, true, true, false, false, true, false, false, false,
+/* STRING */ true, false, true, false, false, false, false, true, false, false, false,
+/* NUMBER */ true, false, false, true, false, false, false, true, false, false, false,
+/* INTEGER */ true, false, false, true, true, false, false, true, false, false, false,
+/* BOOLEAN */ true, false, false, false, false, true, false, true, false, false, false,
+/* VARBINARY*/ true, false, false, false, false, false, true, true, false, false, false,
+/* SCALAR */ true, false, false, false, false, false, false, true, false, false, false,
+/* DECIMAL */ true, false, false, true, false, false, false, true, true, false, false,
+/* ARRAY */ true, false, false, false, false, false, false, false, false, true, false,
+/* MAP */ true, false, false, false, false, false, false, false, false, false, true,
};
bool
diff --git a/src/box/field_def.h b/src/box/field_def.h
index 2f21f3cfc..abd8ca28b 100644
--- a/src/box/field_def.h
+++ b/src/box/field_def.h
@@ -59,6 +59,7 @@ enum field_type {
FIELD_TYPE_BOOLEAN,
FIELD_TYPE_VARBINARY,
FIELD_TYPE_SCALAR,
+ FIELD_TYPE_DECIMAL,
FIELD_TYPE_ARRAY,
FIELD_TYPE_MAP,
field_type_MAX
diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
index b4932db6d..990b584c4 100644
--- a/src/box/tuple_compare.cc
+++ b/src/box/tuple_compare.cc
@@ -33,6 +33,9 @@
#include "coll/coll.h"
#include "trivia/util.h" /* NOINLINE */
#include <math.h>
+#include "lib/core/decimal.h"
+#include "lib/core/mp_decimal.h"
+#include "lib/core/mp_user_types.h"
/* {{{ tuple_compare */
@@ -88,7 +91,7 @@ static enum mp_class mp_classes[] = {
/* .MP_FIELD_BOOL = */ MP_CLASS_BOOL,
/* .MP_FIELD_FLOAT = */ MP_CLASS_NUMBER,
/* .MP_FIELD_DOUBLE = */ MP_CLASS_NUMBER,
- /* .MP_FIELD_BIN = */ MP_CLASS_BIN
+ /* .MP_FIELD_DECIMAL = */ MP_CLASS_NUMBER
};
#define COMPARE_RESULT(a, b) (a < b ? -1 : a > b)
@@ -265,6 +268,45 @@ mp_compare_double_any_number(double lhs, const char *rhs,
return k * COMPARE_RESULT(lqbit, rqbit);
}
+static int
+mp_compare_decimal_any_number(decimal_t *lhs, const char *rhs,
+ enum mp_field_type rhs_type, int k)
+{
+ decimal_t rhs_dec;
+ switch (rhs_type) {
+ case MP_FIELD_FLOAT:
+ {
+ double d = mp_decode_float(&rhs);
+ decimal_from_double(&rhs_dec, d);
+ break;
+ }
+ case MP_FIELD_DOUBLE:
+ {
+ double d = mp_decode_double(&rhs);
+ decimal_from_double(&rhs_dec, d);
+ break;
+ }
+ case MP_FIELD_INT:
+ {
+ int64_t num = mp_decode_int(&rhs);
+ decimal_from_int64(&rhs_dec, num);
+ break;
+ }
+ case MP_FIELD_UINT:
+ {
+ uint64_t num = mp_decode_uint(&rhs);
+ decimal_from_uint64(&rhs_dec, num);
+ break;
+ }
+ case MP_FIELD_DECIMAL:
+ mp_decode_decimal(&rhs, &rhs_dec);
+ break;
+ default:
+ unreachable();
+ }
+ return k * decimal_compare(lhs, &rhs_dec);
+}
+
static int
mp_compare_number_with_type(const char *lhs, enum mp_field_type lhs_type,
const char *rhs, enum mp_field_type rhs_type)
@@ -272,6 +314,21 @@ mp_compare_number_with_type(const char *lhs, enum mp_field_type lhs_type,
assert(mp_classof(lhs_type) == MP_CLASS_NUMBER);
assert(mp_classof(rhs_type) == MP_CLASS_NUMBER);
+ /*
+ * Test decimals first, so that we don't have to
+ * account for them in other comparators.
+ */
+ decimal_t dec;
+ if (rhs_type == MP_FIELD_DECIMAL) {
+ return mp_compare_decimal_any_number(
+ mp_decode_decimal(&rhs, &dec), lhs, lhs_type, -1
+ );
+ }
+ if (lhs_type == MP_FIELD_DECIMAL) {
+ return mp_compare_decimal_any_number(
+ mp_decode_decimal(&lhs, &dec), rhs, rhs_type, 1
+ );
+ }
if (rhs_type == MP_FIELD_FLOAT) {
return mp_compare_double_any_number(
mp_decode_float(&rhs), lhs, lhs_type, -1
@@ -412,6 +469,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
return coll != NULL ?
mp_compare_scalar_coll(field_a, field_b, coll) :
mp_compare_scalar(field_a, field_b);
+ case FIELD_TYPE_DECIMAL:
+ return mp_compare_number(field_a, field_b);
default:
unreachable();
return 0;
@@ -445,6 +504,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_field_type a_type,
mp_compare_scalar_coll(field_a, field_b, coll) :
mp_compare_scalar_with_type(field_a, a_type,
field_b, b_type);
+ case FIELD_TYPE_DECIMAL:
+ return mp_compare_number(field_a, field_b);
default:
unreachable();
return 0;
@@ -1504,6 +1565,24 @@ hint_double(double d)
return hint_create(MP_CLASS_NUMBER, val);
}
+static inline hint_t
+hint_decimal(decimal_t *dec)
+{
+ uint64_t val = 0;
+ int64_t num;
+ if (decimal_to_int64(dec, &num) &&
+ num >= HINT_VALUE_INT_MIN && num <= HINT_VALUE_INT_MAX) {
+ val = num - HINT_VALUE_INT_MIN;
+ } else if (!(dec->bits & DECNEG)) {
+ val = HINT_VALUE_MAX;
+ }
+ /*
+ * In case the number is negative and out of bounds, val
+ * remains zero.
+ */
+ return hint_create(MP_CLASS_NUMBER, val);
+}
+
static inline uint64_t
hint_str_raw(const char *s, uint32_t len)
{
@@ -1580,12 +1659,25 @@ field_hint_number(const char *field)
return hint_double(mp_decode_float(&field));
case MP_FIELD_DOUBLE:
return hint_double(mp_decode_double(&field));
+ case MP_FIELD_DECIMAL:
+ {
+ decimal_t dec;
+ return hint_decimal(mp_decode_decimal(&field, &dec));
+ }
default:
unreachable();
}
return HINT_NONE;
}
+static inline hint_t
+field_hint_decimal(const char *field)
+{
+ assert(msgpack_to_field_type(field) == MP_FIELD_DECIMAL);
+ decimal_t dec;
+ return hint_decimal(mp_decode_decimal(&field, &dec));
+}
+
static inline hint_t
field_hint_string(const char *field, struct coll *coll)
{
@@ -1625,6 +1717,11 @@ field_hint_scalar(const char *field, struct coll *coll)
case MP_FIELD_BIN:
len = mp_decode_binl(&field);
return hint_bin(field, len);
+ case MP_FIELD_DECIMAL:
+ {
+ decimal_t dec;
+ return hint_decimal(mp_decode_decimal(&field, &dec));
+ }
default:
unreachable();
}
@@ -1652,6 +1749,8 @@ field_hint(const char *field, struct coll *coll)
return field_hint_varbinary(field);
case FIELD_TYPE_SCALAR:
return field_hint_scalar(field, coll);
+ case FIELD_TYPE_DECIMAL:
+ return field_hint_decimal(field);
default:
unreachable();
}
@@ -1757,6 +1856,9 @@ key_def_set_hint_func(struct key_def *def)
case FIELD_TYPE_SCALAR:
key_def_set_hint_func<FIELD_TYPE_SCALAR>(def);
break;
+ case FIELD_TYPE_DECIMAL:
+ key_def_set_hint_func<FIELD_TYPE_DECIMAL>(def);
+ break;
default:
/* Invalid key definition. */
def->key_hint = NULL;
diff --git a/src/lib/core/decimal.h b/src/lib/core/decimal.h
index cf41a5052..72c9fd4b5 100644
--- a/src/lib/core/decimal.h
+++ b/src/lib/core/decimal.h
@@ -37,6 +37,10 @@
#include "third_party/decNumber/decNumber.h"
#include <stdint.h>
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
typedef decNumber decimal_t;
/**
@@ -233,4 +237,8 @@ decimal_pack(char *data, const decimal_t *dec);
decimal_t *
decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
#endif /* TARANTOOL_LIB_CORE_DECIMAL_H_INCLUDED */
diff --git a/src/lib/core/mp_decimal.h b/src/lib/core/mp_decimal.h
index a991a5f16..778529068 100644
--- a/src/lib/core/mp_decimal.h
+++ b/src/lib/core/mp_decimal.h
@@ -34,6 +34,10 @@
#include "decimal.h"
#include <stdint.h>
+#if defined(__cplusplus)
+extern "C" {
+#endif /* defined(__cplusplus) */
+
/**
* \brief Calculate exact buffer size needed to store a decimal
* pointed to by \a dec.
@@ -59,4 +63,8 @@ mp_decode_decimal(const char **data, decimal_t *dec);
char *
mp_encode_decimal(char *data, const decimal_t *dec);
+#if defined(__cplusplus)
+} /* extern "C" */
+#endif /* defined(__cplusplus) */
+
#endif
diff --git a/test/engine/ddl.result b/test/engine/ddl.result
index fa3f6051f..67b22ed9e 100644
--- a/test/engine/ddl.result
+++ b/test/engine/ddl.result
@@ -1034,6 +1034,9 @@ s:drop()
--
-- gh-2800: space formats checking is broken.
--
+decimal = require('decimal')
+---
+...
-- Ensure that vinyl correctly process field count change.
s = box.schema.space.create('test', {engine = engine, field_count = 2})
---
@@ -1092,13 +1095,16 @@ format[9] = {name = 'field9', type = 'array'}
format[10] = {name = 'field10', type = 'map'}
---
...
+format[11] = {name = 'field11', type = 'decimal'}
+---
+...
s = box.schema.space.create('test', {engine = engine, format = format})
---
...
pk = s:create_index('pk')
---
...
-t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}}
+t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}, decimal.new(-11.11)}
---
...
inspector:cmd("setopt delimiter ';'")
@@ -1160,6 +1166,11 @@ fail_format_change(3, 'map')
---
- 'Tuple field 3 type does not match one required by operation: expected map'
...
+-- unsigned --X--> decimal
+fail_format_change(3, 'decimal')
+---
+- 'Tuple field 3 type does not match one required by operation: expected decimal'
+...
-- string -----> any
ok_format_change(4, 'any')
---
@@ -1173,6 +1184,11 @@ fail_format_change(4, 'boolean')
---
- 'Tuple field 4 type does not match one required by operation: expected boolean'
...
+-- string --X--> decimal
+fail_format_change(4, 'decimal')
+---
+- 'Tuple field 4 type does not match one required by operation: expected decimal'
+...
-- number -----> any
ok_format_change(5, 'any')
---
@@ -1186,6 +1202,11 @@ fail_format_change(5, 'integer')
---
- 'Tuple field 5 type does not match one required by operation: expected integer'
...
+-- number --X--> decimal
+fail_format_change(5, 'decimal')
+---
+- 'Tuple field 5 type does not match one required by operation: expected decimal'
+...
-- integer -----> any
ok_format_change(6, 'any')
---
@@ -1203,6 +1224,11 @@ fail_format_change(6, 'unsigned')
---
- 'Tuple field 6 type does not match one required by operation: expected unsigned'
...
+-- integer --X--> decimal
+fail_format_change(6, 'decimal')
+---
+- 'Tuple field 6 type does not match one required by operation: expected decimal'
+...
-- boolean -----> any
ok_format_change(7, 'any')
---
@@ -1216,6 +1242,11 @@ fail_format_change(7, 'string')
---
- 'Tuple field 7 type does not match one required by operation: expected string'
...
+-- boolead --X--> decimal
+fail_format_change(7, 'decimal')
+---
+- 'Tuple field 7 type does not match one required by operation: expected decimal'
+...
-- scalar -----> any
ok_format_change(8, 'any')
---
@@ -1225,6 +1256,11 @@ fail_format_change(8, 'unsigned')
---
- 'Tuple field 8 type does not match one required by operation: expected unsigned'
...
+-- scalar --X--> decimal
+fail_format_change(8, 'decimal')
+---
+- 'Tuple field 8 type does not match one required by operation: expected decimal'
+...
-- array -----> any
ok_format_change(9, 'any')
---
@@ -1234,6 +1270,11 @@ fail_format_change(9, 'scalar')
---
- 'Tuple field 9 type does not match one required by operation: expected scalar'
...
+-- arary --X--> decimal
+fail_format_change(9, 'decimal')
+---
+- 'Tuple field 9 type does not match one required by operation: expected decimal'
+...
-- map -----> any
ok_format_change(10, 'any')
---
@@ -1243,6 +1284,48 @@ fail_format_change(10, 'scalar')
---
- 'Tuple field 10 type does not match one required by operation: expected scalar'
...
+-- map --X--> decimal
+fail_format_change(10, 'decimal')
+---
+- 'Tuple field 10 type does not match one required by operation: expected decimal'
+...
+-- decimal ----> any
+ok_format_change(11, 'any')
+---
+...
+-- decimal ----> number
+ok_format_change(11, 'number')
+---
+...
+-- decimal ----> scalar
+ok_format_change(11, 'scalar')
+---
+...
+-- decimal --X--> string
+fail_format_change(11, 'string')
+---
+- 'Tuple field 11 type does not match one required by operation: expected string'
+...
+-- decimal --X--> integer
+fail_format_change(11, 'integer')
+---
+- 'Tuple field 11 type does not match one required by operation: expected integer'
+...
+-- decimal --X--> unsigned
+fail_format_change(11, 'unsigned')
+---
+- 'Tuple field 11 type does not match one required by operation: expected unsigned'
+...
+-- decimal --X--> map
+fail_format_change(11, 'map')
+---
+- 'Tuple field 11 type does not match one required by operation: expected map'
+...
+-- decimal --X--> array
+fail_format_change(11, 'array')
+---
+- 'Tuple field 11 type does not match one required by operation: expected array'
+...
s:drop()
---
...
diff --git a/test/engine/ddl.test.lua b/test/engine/ddl.test.lua
index d15bf1f58..e761966d7 100644
--- a/test/engine/ddl.test.lua
+++ b/test/engine/ddl.test.lua
@@ -355,6 +355,8 @@ s:drop()
-- gh-2800: space formats checking is broken.
--
+decimal = require('decimal')
+
-- Ensure that vinyl correctly process field count change.
s = box.schema.space.create('test', {engine = engine, field_count = 2})
pk = s:create_index('pk')
@@ -376,9 +378,11 @@ format[7] = {name = 'field7', type = 'boolean'}
format[8] = {name = 'field8', type = 'scalar'}
format[9] = {name = 'field9', type = 'array'}
format[10] = {name = 'field10', type = 'map'}
+format[11] = {name = 'field11', type = 'decimal'}
+
s = box.schema.space.create('test', {engine = engine, format = format})
pk = s:create_index('pk')
-t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}}
+t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}, decimal.new(-11.11)}
inspector:cmd("setopt delimiter ';'")
function fail_format_change(fieldno, new_type)
@@ -415,6 +419,8 @@ ok_format_change(3, 'integer')
ok_format_change(3, 'scalar')
-- unsigned --X--> map
fail_format_change(3, 'map')
+-- unsigned --X--> decimal
+fail_format_change(3, 'decimal')
-- string -----> any
ok_format_change(4, 'any')
@@ -422,6 +428,8 @@ ok_format_change(4, 'any')
ok_format_change(4, 'scalar')
-- string --X--> boolean
fail_format_change(4, 'boolean')
+-- string --X--> decimal
+fail_format_change(4, 'decimal')
-- number -----> any
ok_format_change(5, 'any')
@@ -429,6 +437,8 @@ ok_format_change(5, 'any')
ok_format_change(5, 'scalar')
-- number --X--> integer
fail_format_change(5, 'integer')
+-- number --X--> decimal
+fail_format_change(5, 'decimal')
-- integer -----> any
ok_format_change(6, 'any')
@@ -438,6 +448,8 @@ ok_format_change(6, 'number')
ok_format_change(6, 'scalar')
-- integer --X--> unsigned
fail_format_change(6, 'unsigned')
+-- integer --X--> decimal
+fail_format_change(6, 'decimal')
-- boolean -----> any
ok_format_change(7, 'any')
@@ -445,22 +457,46 @@ ok_format_change(7, 'any')
ok_format_change(7, 'scalar')
-- boolean --X--> string
fail_format_change(7, 'string')
+-- boolead --X--> decimal
+fail_format_change(7, 'decimal')
-- scalar -----> any
ok_format_change(8, 'any')
-- scalar --X--> unsigned
fail_format_change(8, 'unsigned')
+-- scalar --X--> decimal
+fail_format_change(8, 'decimal')
-- array -----> any
ok_format_change(9, 'any')
-- array --X--> scalar
fail_format_change(9, 'scalar')
+-- arary --X--> decimal
+fail_format_change(9, 'decimal')
-- map -----> any
ok_format_change(10, 'any')
-- map --X--> scalar
fail_format_change(10, 'scalar')
-
+-- map --X--> decimal
+fail_format_change(10, 'decimal')
+
+-- decimal ----> any
+ok_format_change(11, 'any')
+-- decimal ----> number
+ok_format_change(11, 'number')
+-- decimal ----> scalar
+ok_format_change(11, 'scalar')
+-- decimal --X--> string
+fail_format_change(11, 'string')
+-- decimal --X--> integer
+fail_format_change(11, 'integer')
+-- decimal --X--> unsigned
+fail_format_change(11, 'unsigned')
+-- decimal --X--> map
+fail_format_change(11, 'map')
+-- decimal --X--> array
+fail_format_change(11, 'array')
s:drop()
-- Check new fields adding.
diff --git a/test/engine/decimal.result b/test/engine/decimal.result
index f8888d7c9..4ced88c24 100644
--- a/test/engine/decimal.result
+++ b/test/engine/decimal.result
@@ -12,6 +12,9 @@ engine = test_run:get_cfg('engine')
decimal = require('decimal')
| ---
| ...
+ffi = require('ffi')
+ | ---
+ | ...
_ = box.schema.space.create('test', {engine=engine})
| ---
@@ -76,6 +79,278 @@ box.space.test:select{}
| - [3, 3.3]
| - [4, 1234567890123456789.9876543210987654321, 1.2345]
| ...
+
+box.space.test:drop()
+ | ---
+ | ...
+
+-- check decimal indexes
+_ = box.schema.space.create('test', {engine=engine})
+ | ---
+ | ...
+_ = box.space.test:create_index('pk', {parts={1,'decimal'}})
+ | ---
+ | ...
+
+test_run:cmd('setopt delimiter ";"')
+ | ---
+ | - true
+ | ...
+for i = 0,16 do
+ box.space.test:insert{decimal.new((i-8)/4)}
+end;
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ""');
+ | ---
+ | - true
+ | ...
+
+box.space.test:select{}
+ | ---
+ | - - [-2]
+ | - [-1.75]
+ | - [-1.5]
+ | - [-1.25]
+ | - [-1]
+ | - [-0.75]
+ | - [-0.5]
+ | - [-0.25]
+ | - [0]
+ | - [0.25]
+ | - [0.5]
+ | - [0.75]
+ | - [1]
+ | - [1.25]
+ | - [1.5]
+ | - [1.75]
+ | - [2]
+ | ...
+
+-- check invalid values
+box.space.test:insert{1.23}
+ | ---
+ | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
+ | ...
+box.space.test:insert{'str'}
+ | ---
+ | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
+ | ...
+box.space.test:insert{ffi.new('uint64_t', 0)}
+ | ---
+ | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
+ | ...
+-- check duplicates
+box.space.test:insert{decimal.new(0)}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+
+box.space.test.index.pk:drop()
+ | ---
+ | ...
+
+_ = box.space.test:create_index('pk', {parts={1, 'number'}})
+ | ---
+ | ...
+
+test_run:cmd('setopt delimiter ";"')
+ | ---
+ | - true
+ | ...
+for i = 0, 32 do
+ local val = (i - 16) / 8
+ if i % 2 == 1 then val = decimal.new(val) end
+ box.space.test:insert{val}
+end;
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ""');
+ | ---
+ | - true
+ | ...
+
+box.space.test:select{}
+ | ---
+ | - - [-2]
+ | - [-1.875]
+ | - [-1.75]
+ | - [-1.625]
+ | - [-1.5]
+ | - [-1.375]
+ | - [-1.25]
+ | - [-1.125]
+ | - [-1]
+ | - [-0.875]
+ | - [-0.75]
+ | - [-0.625]
+ | - [-0.5]
+ | - [-0.375]
+ | - [-0.25]
+ | - [-0.125]
+ | - [0]
+ | - [0.125]
+ | - [0.25]
+ | - [0.375]
+ | - [0.5]
+ | - [0.625]
+ | - [0.75]
+ | - [0.875]
+ | - [1]
+ | - [1.125]
+ | - [1.25]
+ | - [1.375]
+ | - [1.5]
+ | - [1.625]
+ | - [1.75]
+ | - [1.875]
+ | - [2]
+ | ...
+
+-- check duplicates
+box.space.test:insert{-2}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+box.space.test:insert{decimal.new(-2)}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+box.space.test:insert{decimal.new(-1.875)}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+box.space.test:insert{-1.875}
+ | ---
+ | - error: Duplicate key exists in unique index 'pk' in space 'test'
+ | ...
+
+box.space.test.index.pk:drop()
+ | ---
+ | ...
+
+_ = box.space.test:create_index('pk')
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ";"')
+ | ---
+ | - true
+ | ...
+for i = 1,10 do
+ box.space.test:insert{i, decimal.new(i/10)}
+end;
+ | ---
+ | ...
+test_run:cmd('setopt delimiter ""');
+ | ---
+ | - true
+ | ...
+
+-- a bigger test with a secondary index this time.
+box.space.test:insert{11, 'str'}
+ | ---
+ | - [11, 'str']
+ | ...
+box.space.test:insert{12, 0.63}
+ | ---
+ | - [12, 0.63]
+ | ...
+box.space.test:insert{13, 0.57}
+ | ---
+ | - [13, 0.57]
+ | ...
+box.space.test:insert{14, 0.33}
+ | ---
+ | - [14, 0.33]
+ | ...
+box.space.test:insert{16, 0.71}
+ | ---
+ | - [16, 0.71]
+ | ...
+
+_ = box.space.test:create_index('sk', {parts={2, 'scalar'}})
+ | ---
+ | ...
+box.space.test.index.sk:select{}
+ | ---
+ | - - [1, 0.1]
+ | - [2, 0.2]
+ | - [3, 0.3]
+ | - [14, 0.33]
+ | - [4, 0.4]
+ | - [5, 0.5]
+ | - [13, 0.57]
+ | - [6, 0.6]
+ | - [12, 0.63]
+ | - [7, 0.7]
+ | - [16, 0.71]
+ | - [8, 0.8]
+ | - [9, 0.9]
+ | - [10, 1]
+ | - [11, 'str']
+ | ...
+
+box.space.test:drop()
+ | ---
+ | ...
+
+-- check index alter
+_ = box.schema.space.create('test', {engine=engine})
+ | ---
+ | ...
+_ = box.space.test:create_index('pk')
+ | ---
+ | ...
+_ = box.space.test:create_index('sk', {parts={2, 'number'}})
+ | ---
+ | ...
+box.space.test:insert{1, decimal.new(-2)}
+ | ---
+ | - [1, -2]
+ | ...
+box.space.test:insert{2, -5}
+ | ---
+ | - [2, -5]
+ | ...
+-- failure
+box.space.test.index.sk:alter{parts={2, 'decimal'}}
+ | ---
+ | - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
+ | ...
+box.space.test:delete{2}
+ | ---
+ | ...
+box.space.test.index.sk:alter{parts={2, 'decimal'}}
+ | ---
+ | ...
+box.space.test:insert{3, decimal.new(3)}
+ | ---
+ | - [3, 3]
+ | ...
+--failure
+box.space.test:insert{4, 'string'}
+ | ---
+ | - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
+ | ...
+-- failure
+box.space.test:insert{2, -5}
+ | ---
+ | - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
+ | ...
+box.space.test.index.sk:alter{parts={2, 'number'}}
+ | ---
+ | ...
+box.space.test:insert{2, -5}
+ | ---
+ | - [2, -5]
+ | ...
+box.space.test.index.sk:select{}
+ | ---
+ | - - [2, -5]
+ | - [1, -2]
+ | - [3, 3]
+ | ...
+
box.space.test:drop()
| ---
| ...
diff --git a/test/engine/decimal.test.lua b/test/engine/decimal.test.lua
index 1b14871b0..4e38650dd 100644
--- a/test/engine/decimal.test.lua
+++ b/test/engine/decimal.test.lua
@@ -3,6 +3,7 @@ test_run = env.new()
engine = test_run:get_cfg('engine')
decimal = require('decimal')
+ffi = require('ffi')
_ = box.schema.space.create('test', {engine=engine})
_ = box.space.test:create_index('pk')
@@ -18,4 +19,86 @@ a:next(2)
a:slice(-2)
box.space.test:replace{3, decimal.new(3.3)}
box.space.test:select{}
+
+box.space.test:drop()
+
+-- check decimal indexes
+_ = box.schema.space.create('test', {engine=engine})
+_ = box.space.test:create_index('pk', {parts={1,'decimal'}})
+
+test_run:cmd('setopt delimiter ";"')
+for i = 0,16 do
+ box.space.test:insert{decimal.new((i-8)/4)}
+end;
+test_run:cmd('setopt delimiter ""');
+
+box.space.test:select{}
+
+-- check invalid values
+box.space.test:insert{1.23}
+box.space.test:insert{'str'}
+box.space.test:insert{ffi.new('uint64_t', 0)}
+-- check duplicates
+box.space.test:insert{decimal.new(0)}
+
+box.space.test.index.pk:drop()
+
+_ = box.space.test:create_index('pk', {parts={1, 'number'}})
+
+test_run:cmd('setopt delimiter ";"')
+for i = 0, 32 do
+ local val = (i - 16) / 8
+ if i % 2 == 1 then val = decimal.new(val) end
+ box.space.test:insert{val}
+end;
+test_run:cmd('setopt delimiter ""');
+
+box.space.test:select{}
+
+-- check duplicates
+box.space.test:insert{-2}
+box.space.test:insert{decimal.new(-2)}
+box.space.test:insert{decimal.new(-1.875)}
+box.space.test:insert{-1.875}
+
+box.space.test.index.pk:drop()
+
+_ = box.space.test:create_index('pk')
+test_run:cmd('setopt delimiter ";"')
+for i = 1,10 do
+ box.space.test:insert{i, decimal.new(i/10)}
+end;
+test_run:cmd('setopt delimiter ""');
+
+-- a bigger test with a secondary index this time.
+box.space.test:insert{11, 'str'}
+box.space.test:insert{12, 0.63}
+box.space.test:insert{13, 0.57}
+box.space.test:insert{14, 0.33}
+box.space.test:insert{16, 0.71}
+
+_ = box.space.test:create_index('sk', {parts={2, 'scalar'}})
+box.space.test.index.sk:select{}
+
+box.space.test:drop()
+
+-- check index alter
+_ = box.schema.space.create('test', {engine=engine})
+_ = box.space.test:create_index('pk')
+_ = box.space.test:create_index('sk', {parts={2, 'number'}})
+box.space.test:insert{1, decimal.new(-2)}
+box.space.test:insert{2, -5}
+-- failure
+box.space.test.index.sk:alter{parts={2, 'decimal'}}
+box.space.test:delete{2}
+box.space.test.index.sk:alter{parts={2, 'decimal'}}
+box.space.test:insert{3, decimal.new(3)}
+--failure
+box.space.test:insert{4, 'string'}
+-- failure
+box.space.test:insert{2, -5}
+box.space.test.index.sk:alter{parts={2, 'number'}}
+box.space.test:insert{2, -5}
+box.space.test.index.sk:select{}
+
box.space.test:drop()
--
2.20.1 (Apple Git-117)
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 8/8] decimal: allow to index decimals
2019-08-08 11:55 ` [PATCH v2 8/8] decimal: allow to index decimals Serge Petrenko
@ 2019-08-08 13:42 ` Serge Petrenko
2019-08-12 21:41 ` Konstantin Osipov
1 sibling, 0 replies; 35+ messages in thread
From: Serge Petrenko @ 2019-08-08 13:42 UTC (permalink / raw)
To: Vladimir Davydov; +Cc: tarantool-patches, kostja
Sorry, a tiny fix for test. Didn’t notice the problem on my local machine.
diff --git a/test/engine/decimal.result b/test/engine/decimal.result
index 4ced88c24..415868c89 100644
--- a/test/engine/decimal.result
+++ b/test/engine/decimal.result
@@ -317,7 +317,7 @@ box.space.test.index.sk:alter{parts={2, 'decimal'}}
| ---
| - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
| ...
-box.space.test:delete{2}
+_ = box.space.test:delete{2}
| ---
| ...
box.space.test.index.sk:alter{parts={2, 'decimal'}}
diff --git a/test/engine/decimal.test.lua b/test/engine/decimal.test.lua
index 4e38650dd..3763bf0a3 100644
--- a/test/engine/decimal.test.lua
+++ b/test/engine/decimal.test.lua
@@ -90,7 +90,7 @@ box.space.test:insert{1, decimal.new(-2)}
box.space.test:insert{2, -5}
-- failure
box.space.test.index.sk:alter{parts={2, 'decimal'}}
-box.space.test:delete{2}
+_ = box.space.test:delete{2}
box.space.test.index.sk:alter{parts={2, 'decimal'}}
box.space.test:insert{3, decimal.new(3)}
--failure
--
Serge Petrenko
sergepetrenko@tarantool.org
> 8 авг. 2019 г., в 14:55, Serge Petrenko <sergepetrenko@tarantool.org> написал(а):
>
> Closes #4333
>
> @TarantoolBot document
> Title: Document decimal field type.
>
> Decimals may now be stored in spaces. A corresponding field type is
> introduced: 'decimal'. Decimal values are also allowed in 'scalar',
> 'any' and 'number' fields.
>
> 'decimal' field type is appropriate for both memtx HASH and TREE
> indices, as well as for vinyl TREE index.
>
> ```
> To create an index 'pk' over a decimal field, say
> ```
> tarantool> box.space.test:create_index('pk', {parts={1, 'decimal'}})
> ---
> - unique: true
> parts:
> - type: decimal
> is_nullable: false
> fieldno: 1
> id: 0
> space_id: 512
> type: TREE
> name: pk
> ...
> ```
> Now you can insert some decimal values:
> ```
> tarantool> for i = 1,10 do
>> box.space.test:insert{decimal.new((i-5)/10)}
>> end
> ---
> ...
> ```
> tarantool> box.space.test:select{}
> ---
> - - [-0.4]
> - [-0.3]
> - [-0.2]
> - [-0.1]
> - [0]
> - [0.1]
> - [0.2]
> - [0.3]
> - [0.4]
> - [0.5]
> ...
> ```
> Decimals may alse be inserted into `scalar` and `number` fields. In this
> case all the number values are sorted correctly:
> ```
> tarantool> box.schema.space.create('test')
> tarantool> box.space.test:create_index('pk', {parts={1, 'number'}})
> tarantool> box.space.test:insert{-1.0001, 'number'}
> ---
> - [-1.0001, 'number']
> ...
>
> tarantool> box.space.test:insert{decimal.new(-1.00001), 'decimal'}
> ---
> - [-1.00001, 'decimal']
> ...
>
> tarantool> box.space.test:insert{-1, 'number'}
> ---
> - [-1, 'number']
> ...
>
> tarantool> box.space.test:insert{decimal.new(-0.999), 'decimal'}
> ---
> - [-0.999, 'decimal']
> ...
>
> tarantool> box.space.test:insert{-0.998, 'number'}
> ---
> - [-0.998, 'number']
> ...
>
> tarantool> box.space.test:insert{-0.9, 'number'}
> ---
> - [-0.9, 'number']
> ...
>
> tarantool> box.space.test:insert{-0.95, 'number'}
> ---
> - [-0.95, 'number']
> ...
>
> tarantool> box.space.test:insert{decimal.new(-0.92), 'decimal'}
> ---
> - [-0.92, 'decimal']
> ...
>
> tarantool> box.space.test:insert{decimal.new(-0.971), 'decimal'}
> ---
> - [-0.971, 'decimal']
> ...
>
> tarantool> box.space.test:select{}
> ---
> - - [-1.0001, 'number']
> - [-1.00001, 'decimal']
> - [-1, 'number']
> - [-0.999, 'decimal']
> - [-0.998, 'number']
> - [-0.971, 'decimal']
> - [-0.95, 'number']
> - [-0.92, 'decimal']
> - [-0.9, 'number']
> ...
>
> ```
> Uniqueness is also preserved between decimals and other number types:
> ```
> tarantool> box.space.test:insert{-0.92}
> ---
> - error: Duplicate key exists in unique index 'pk' in space 'test'
> ...
>
> tarantool> box.space.test:insert{decimal.new(-0.9)}
> ---
> - error: Duplicate key exists in unique index 'pk' in space 'test'
> ...
>
> ```
>
> You can also set decimal fields in space format:
> ```
> tarantool> _ = box.schema.space.create('test')
> ---
> ...
>
> tarantool> _ = box.space.test:create_index('pk')
> ---
> ...
>
> tarantool> box.space.test:format{{name='id', type='unsigned'}, {name='balance', type='decimal'}}
> ---
> ...
>
> tarantool> box.space.test:insert{1}
> ---
> - error: Tuple field 2 required by space format is missing
> ...
>
> tarantool> box.space.test:insert{1, 'string'}
> ---
> - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
> ...
>
> tarantool> box.space.test:insert{1, 1.2345}
> ---
> - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
> ...
>
> tarantool> box.space.test:insert{1, decimal.new('1337.420')}
> ---
> - [1, 1337.420]
> ...
>
> ```
> ---
> src/box/field_def.c | 30 ++--
> src/box/field_def.h | 1 +
> src/box/tuple_compare.cc | 104 ++++++++++++-
> src/lib/core/decimal.h | 8 +
> src/lib/core/mp_decimal.h | 8 +
> test/engine/ddl.result | 85 ++++++++++-
> test/engine/ddl.test.lua | 40 ++++-
> test/engine/decimal.result | 275 +++++++++++++++++++++++++++++++++++
> test/engine/decimal.test.lua | 83 +++++++++++
> 9 files changed, 617 insertions(+), 17 deletions(-)
>
> diff --git a/src/box/field_def.c b/src/box/field_def.c
> index 2fad81d42..b1ecb1f19 100644
> --- a/src/box/field_def.c
> +++ b/src/box/field_def.c
> @@ -32,6 +32,7 @@
> #include "field_def.h"
> #include "trivia/util.h"
> #include "key_def.h"
> +#include "lua/utils.h"
>
> const char *mp_type_strs[] = {
> /* .MP_NIL = */ "nil",
> @@ -52,13 +53,14 @@ const uint32_t field_mp_type[] = {
> /* [FIELD_TYPE_UNSIGNED] = */ 1U << MP_FIELD_UINT,
> /* [FIELD_TYPE_STRING] = */ 1U << MP_FIELD_STR,
> /* [FIELD_TYPE_NUMBER] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT) |
> - (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE),
> + (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE) | (1U << MP_FIELD_DECIMAL),
> /* [FIELD_TYPE_INTEGER] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT),
> /* [FIELD_TYPE_BOOLEAN] = */ 1U << MP_FIELD_BOOL,
> /* [FIELD_TYPE_VARBINARY] = */ 1U << MP_FIELD_BIN,
> /* [FIELD_TYPE_SCALAR] = */ (1U << MP_FIELD_UINT) | (1U << MP_FIELD_INT) |
> (1U << MP_FIELD_FLOAT) | (1U << MP_FIELD_DOUBLE) | (1U << MP_FIELD_STR) |
> - (1U << MP_FIELD_BIN) | (1U << MP_FIELD_BOOL),
> + (1U << MP_FIELD_BIN) | (1U << MP_FIELD_BOOL) | (1U << MP_FIELD_DECIMAL),
> + /* [FIELD_TYPE_DECIMAL] = */ 1U << MP_FIELD_DECIMAL,
> /* [FIELD_TYPE_ARRAY] = */ 1U << MP_FIELD_ARRAY,
> /* [FIELD_TYPE_MAP] = */ (1U << MP_FIELD_MAP),
> };
> @@ -72,6 +74,7 @@ const char *field_type_strs[] = {
> /* [FIELD_TYPE_BOOLEAN] = */ "boolean",
> /* [FIELD_TYPE_VARBINARY] = */"varbinary",
> /* [FIELD_TYPE_SCALAR] = */ "scalar",
> + /* [FIELD_TYPE_DECIMAL] = */ "decimal",
> /* [FIELD_TYPE_ARRAY] = */ "array",
> /* [FIELD_TYPE_MAP] = */ "map",
> };
> @@ -98,17 +101,18 @@ field_type_by_name_wrapper(const char *str, uint32_t len)
> * values can be stored in the j type.
> */
> static const bool field_type_compatibility[] = {
> - /* ANY UNSIGNED STRING NUMBER INTEGER BOOLEAN VARBINARY SCALAR ARRAY MAP */
> -/* ANY */ true, false, false, false, false, false, false, false, false, false,
> -/* UNSIGNED */ true, true, false, true, true, false, false, true, false, false,
> -/* STRING */ true, false, true, false, false, false, false, true, false, false,
> -/* NUMBER */ true, false, false, true, false, false, false, true, false, false,
> -/* INTEGER */ true, false, false, true, true, false, false, true, false, false,
> -/* BOOLEAN */ true, false, false, false, false, true, false, true, false, false,
> -/* VARBINARY*/ true, false, false, false, false, false, true, true, false, false,
> -/* SCALAR */ true, false, false, false, false, false, false, true, false, false,
> -/* ARRAY */ true, false, false, false, false, false, false, false, true, false,
> -/* MAP */ true, false, false, false, false, false, false, false, false, true,
> + /* ANY UNSIGNED STRING NUMBER INTEGER BOOLEAN VARBINARY SCALAR DECIMAL ARRAY MAP */
> +/* ANY */ true, false, false, false, false, false, false, false, false, false, false,
> +/* UNSIGNED */ true, true, false, true, true, false, false, true, false, false, false,
> +/* STRING */ true, false, true, false, false, false, false, true, false, false, false,
> +/* NUMBER */ true, false, false, true, false, false, false, true, false, false, false,
> +/* INTEGER */ true, false, false, true, true, false, false, true, false, false, false,
> +/* BOOLEAN */ true, false, false, false, false, true, false, true, false, false, false,
> +/* VARBINARY*/ true, false, false, false, false, false, true, true, false, false, false,
> +/* SCALAR */ true, false, false, false, false, false, false, true, false, false, false,
> +/* DECIMAL */ true, false, false, true, false, false, false, true, true, false, false,
> +/* ARRAY */ true, false, false, false, false, false, false, false, false, true, false,
> +/* MAP */ true, false, false, false, false, false, false, false, false, false, true,
> };
>
> bool
> diff --git a/src/box/field_def.h b/src/box/field_def.h
> index 2f21f3cfc..abd8ca28b 100644
> --- a/src/box/field_def.h
> +++ b/src/box/field_def.h
> @@ -59,6 +59,7 @@ enum field_type {
> FIELD_TYPE_BOOLEAN,
> FIELD_TYPE_VARBINARY,
> FIELD_TYPE_SCALAR,
> + FIELD_TYPE_DECIMAL,
> FIELD_TYPE_ARRAY,
> FIELD_TYPE_MAP,
> field_type_MAX
> diff --git a/src/box/tuple_compare.cc b/src/box/tuple_compare.cc
> index b4932db6d..990b584c4 100644
> --- a/src/box/tuple_compare.cc
> +++ b/src/box/tuple_compare.cc
> @@ -33,6 +33,9 @@
> #include "coll/coll.h"
> #include "trivia/util.h" /* NOINLINE */
> #include <math.h>
> +#include "lib/core/decimal.h"
> +#include "lib/core/mp_decimal.h"
> +#include "lib/core/mp_user_types.h"
>
> /* {{{ tuple_compare */
>
> @@ -88,7 +91,7 @@ static enum mp_class mp_classes[] = {
> /* .MP_FIELD_BOOL = */ MP_CLASS_BOOL,
> /* .MP_FIELD_FLOAT = */ MP_CLASS_NUMBER,
> /* .MP_FIELD_DOUBLE = */ MP_CLASS_NUMBER,
> - /* .MP_FIELD_BIN = */ MP_CLASS_BIN
> + /* .MP_FIELD_DECIMAL = */ MP_CLASS_NUMBER
> };
>
> #define COMPARE_RESULT(a, b) (a < b ? -1 : a > b)
> @@ -265,6 +268,45 @@ mp_compare_double_any_number(double lhs, const char *rhs,
> return k * COMPARE_RESULT(lqbit, rqbit);
> }
>
> +static int
> +mp_compare_decimal_any_number(decimal_t *lhs, const char *rhs,
> + enum mp_field_type rhs_type, int k)
> +{
> + decimal_t rhs_dec;
> + switch (rhs_type) {
> + case MP_FIELD_FLOAT:
> + {
> + double d = mp_decode_float(&rhs);
> + decimal_from_double(&rhs_dec, d);
> + break;
> + }
> + case MP_FIELD_DOUBLE:
> + {
> + double d = mp_decode_double(&rhs);
> + decimal_from_double(&rhs_dec, d);
> + break;
> + }
> + case MP_FIELD_INT:
> + {
> + int64_t num = mp_decode_int(&rhs);
> + decimal_from_int64(&rhs_dec, num);
> + break;
> + }
> + case MP_FIELD_UINT:
> + {
> + uint64_t num = mp_decode_uint(&rhs);
> + decimal_from_uint64(&rhs_dec, num);
> + break;
> + }
> + case MP_FIELD_DECIMAL:
> + mp_decode_decimal(&rhs, &rhs_dec);
> + break;
> + default:
> + unreachable();
> + }
> + return k * decimal_compare(lhs, &rhs_dec);
> +}
> +
> static int
> mp_compare_number_with_type(const char *lhs, enum mp_field_type lhs_type,
> const char *rhs, enum mp_field_type rhs_type)
> @@ -272,6 +314,21 @@ mp_compare_number_with_type(const char *lhs, enum mp_field_type lhs_type,
> assert(mp_classof(lhs_type) == MP_CLASS_NUMBER);
> assert(mp_classof(rhs_type) == MP_CLASS_NUMBER);
>
> + /*
> + * Test decimals first, so that we don't have to
> + * account for them in other comparators.
> + */
> + decimal_t dec;
> + if (rhs_type == MP_FIELD_DECIMAL) {
> + return mp_compare_decimal_any_number(
> + mp_decode_decimal(&rhs, &dec), lhs, lhs_type, -1
> + );
> + }
> + if (lhs_type == MP_FIELD_DECIMAL) {
> + return mp_compare_decimal_any_number(
> + mp_decode_decimal(&lhs, &dec), rhs, rhs_type, 1
> + );
> + }
> if (rhs_type == MP_FIELD_FLOAT) {
> return mp_compare_double_any_number(
> mp_decode_float(&rhs), lhs, lhs_type, -1
> @@ -412,6 +469,8 @@ tuple_compare_field(const char *field_a, const char *field_b,
> return coll != NULL ?
> mp_compare_scalar_coll(field_a, field_b, coll) :
> mp_compare_scalar(field_a, field_b);
> + case FIELD_TYPE_DECIMAL:
> + return mp_compare_number(field_a, field_b);
> default:
> unreachable();
> return 0;
> @@ -445,6 +504,8 @@ tuple_compare_field_with_type(const char *field_a, enum mp_field_type a_type,
> mp_compare_scalar_coll(field_a, field_b, coll) :
> mp_compare_scalar_with_type(field_a, a_type,
> field_b, b_type);
> + case FIELD_TYPE_DECIMAL:
> + return mp_compare_number(field_a, field_b);
> default:
> unreachable();
> return 0;
> @@ -1504,6 +1565,24 @@ hint_double(double d)
> return hint_create(MP_CLASS_NUMBER, val);
> }
>
> +static inline hint_t
> +hint_decimal(decimal_t *dec)
> +{
> + uint64_t val = 0;
> + int64_t num;
> + if (decimal_to_int64(dec, &num) &&
> + num >= HINT_VALUE_INT_MIN && num <= HINT_VALUE_INT_MAX) {
> + val = num - HINT_VALUE_INT_MIN;
> + } else if (!(dec->bits & DECNEG)) {
> + val = HINT_VALUE_MAX;
> + }
> + /*
> + * In case the number is negative and out of bounds, val
> + * remains zero.
> + */
> + return hint_create(MP_CLASS_NUMBER, val);
> +}
> +
> static inline uint64_t
> hint_str_raw(const char *s, uint32_t len)
> {
> @@ -1580,12 +1659,25 @@ field_hint_number(const char *field)
> return hint_double(mp_decode_float(&field));
> case MP_FIELD_DOUBLE:
> return hint_double(mp_decode_double(&field));
> + case MP_FIELD_DECIMAL:
> + {
> + decimal_t dec;
> + return hint_decimal(mp_decode_decimal(&field, &dec));
> + }
> default:
> unreachable();
> }
> return HINT_NONE;
> }
>
> +static inline hint_t
> +field_hint_decimal(const char *field)
> +{
> + assert(msgpack_to_field_type(field) == MP_FIELD_DECIMAL);
> + decimal_t dec;
> + return hint_decimal(mp_decode_decimal(&field, &dec));
> +}
> +
> static inline hint_t
> field_hint_string(const char *field, struct coll *coll)
> {
> @@ -1625,6 +1717,11 @@ field_hint_scalar(const char *field, struct coll *coll)
> case MP_FIELD_BIN:
> len = mp_decode_binl(&field);
> return hint_bin(field, len);
> + case MP_FIELD_DECIMAL:
> + {
> + decimal_t dec;
> + return hint_decimal(mp_decode_decimal(&field, &dec));
> + }
> default:
> unreachable();
> }
> @@ -1652,6 +1749,8 @@ field_hint(const char *field, struct coll *coll)
> return field_hint_varbinary(field);
> case FIELD_TYPE_SCALAR:
> return field_hint_scalar(field, coll);
> + case FIELD_TYPE_DECIMAL:
> + return field_hint_decimal(field);
> default:
> unreachable();
> }
> @@ -1757,6 +1856,9 @@ key_def_set_hint_func(struct key_def *def)
> case FIELD_TYPE_SCALAR:
> key_def_set_hint_func<FIELD_TYPE_SCALAR>(def);
> break;
> + case FIELD_TYPE_DECIMAL:
> + key_def_set_hint_func<FIELD_TYPE_DECIMAL>(def);
> + break;
> default:
> /* Invalid key definition. */
> def->key_hint = NULL;
> diff --git a/src/lib/core/decimal.h b/src/lib/core/decimal.h
> index cf41a5052..72c9fd4b5 100644
> --- a/src/lib/core/decimal.h
> +++ b/src/lib/core/decimal.h
> @@ -37,6 +37,10 @@
> #include "third_party/decNumber/decNumber.h"
> #include <stdint.h>
>
> +#if defined(__cplusplus)
> +extern "C" {
> +#endif /* defined(__cplusplus) */
> +
> typedef decNumber decimal_t;
>
> /**
> @@ -233,4 +237,8 @@ decimal_pack(char *data, const decimal_t *dec);
> decimal_t *
> decimal_unpack(const char **data, uint32_t len, decimal_t *dec);
>
> +#if defined(__cplusplus)
> +} /* extern "C" */
> +#endif /* defined(__cplusplus) */
> +
> #endif /* TARANTOOL_LIB_CORE_DECIMAL_H_INCLUDED */
> diff --git a/src/lib/core/mp_decimal.h b/src/lib/core/mp_decimal.h
> index a991a5f16..778529068 100644
> --- a/src/lib/core/mp_decimal.h
> +++ b/src/lib/core/mp_decimal.h
> @@ -34,6 +34,10 @@
> #include "decimal.h"
> #include <stdint.h>
>
> +#if defined(__cplusplus)
> +extern "C" {
> +#endif /* defined(__cplusplus) */
> +
> /**
> * \brief Calculate exact buffer size needed to store a decimal
> * pointed to by \a dec.
> @@ -59,4 +63,8 @@ mp_decode_decimal(const char **data, decimal_t *dec);
> char *
> mp_encode_decimal(char *data, const decimal_t *dec);
>
> +#if defined(__cplusplus)
> +} /* extern "C" */
> +#endif /* defined(__cplusplus) */
> +
> #endif
> diff --git a/test/engine/ddl.result b/test/engine/ddl.result
> index fa3f6051f..67b22ed9e 100644
> --- a/test/engine/ddl.result
> +++ b/test/engine/ddl.result
> @@ -1034,6 +1034,9 @@ s:drop()
> --
> -- gh-2800: space formats checking is broken.
> --
> +decimal = require('decimal')
> +---
> +...
> -- Ensure that vinyl correctly process field count change.
> s = box.schema.space.create('test', {engine = engine, field_count = 2})
> ---
> @@ -1092,13 +1095,16 @@ format[9] = {name = 'field9', type = 'array'}
> format[10] = {name = 'field10', type = 'map'}
> ---
> ...
> +format[11] = {name = 'field11', type = 'decimal'}
> +---
> +...
> s = box.schema.space.create('test', {engine = engine, format = format})
> ---
> ...
> pk = s:create_index('pk')
> ---
> ...
> -t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}}
> +t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}, decimal.new(-11.11)}
> ---
> ...
> inspector:cmd("setopt delimiter ';'")
> @@ -1160,6 +1166,11 @@ fail_format_change(3, 'map')
> ---
> - 'Tuple field 3 type does not match one required by operation: expected map'
> ...
> +-- unsigned --X--> decimal
> +fail_format_change(3, 'decimal')
> +---
> +- 'Tuple field 3 type does not match one required by operation: expected decimal'
> +...
> -- string -----> any
> ok_format_change(4, 'any')
> ---
> @@ -1173,6 +1184,11 @@ fail_format_change(4, 'boolean')
> ---
> - 'Tuple field 4 type does not match one required by operation: expected boolean'
> ...
> +-- string --X--> decimal
> +fail_format_change(4, 'decimal')
> +---
> +- 'Tuple field 4 type does not match one required by operation: expected decimal'
> +...
> -- number -----> any
> ok_format_change(5, 'any')
> ---
> @@ -1186,6 +1202,11 @@ fail_format_change(5, 'integer')
> ---
> - 'Tuple field 5 type does not match one required by operation: expected integer'
> ...
> +-- number --X--> decimal
> +fail_format_change(5, 'decimal')
> +---
> +- 'Tuple field 5 type does not match one required by operation: expected decimal'
> +...
> -- integer -----> any
> ok_format_change(6, 'any')
> ---
> @@ -1203,6 +1224,11 @@ fail_format_change(6, 'unsigned')
> ---
> - 'Tuple field 6 type does not match one required by operation: expected unsigned'
> ...
> +-- integer --X--> decimal
> +fail_format_change(6, 'decimal')
> +---
> +- 'Tuple field 6 type does not match one required by operation: expected decimal'
> +...
> -- boolean -----> any
> ok_format_change(7, 'any')
> ---
> @@ -1216,6 +1242,11 @@ fail_format_change(7, 'string')
> ---
> - 'Tuple field 7 type does not match one required by operation: expected string'
> ...
> +-- boolead --X--> decimal
> +fail_format_change(7, 'decimal')
> +---
> +- 'Tuple field 7 type does not match one required by operation: expected decimal'
> +...
> -- scalar -----> any
> ok_format_change(8, 'any')
> ---
> @@ -1225,6 +1256,11 @@ fail_format_change(8, 'unsigned')
> ---
> - 'Tuple field 8 type does not match one required by operation: expected unsigned'
> ...
> +-- scalar --X--> decimal
> +fail_format_change(8, 'decimal')
> +---
> +- 'Tuple field 8 type does not match one required by operation: expected decimal'
> +...
> -- array -----> any
> ok_format_change(9, 'any')
> ---
> @@ -1234,6 +1270,11 @@ fail_format_change(9, 'scalar')
> ---
> - 'Tuple field 9 type does not match one required by operation: expected scalar'
> ...
> +-- arary --X--> decimal
> +fail_format_change(9, 'decimal')
> +---
> +- 'Tuple field 9 type does not match one required by operation: expected decimal'
> +...
> -- map -----> any
> ok_format_change(10, 'any')
> ---
> @@ -1243,6 +1284,48 @@ fail_format_change(10, 'scalar')
> ---
> - 'Tuple field 10 type does not match one required by operation: expected scalar'
> ...
> +-- map --X--> decimal
> +fail_format_change(10, 'decimal')
> +---
> +- 'Tuple field 10 type does not match one required by operation: expected decimal'
> +...
> +-- decimal ----> any
> +ok_format_change(11, 'any')
> +---
> +...
> +-- decimal ----> number
> +ok_format_change(11, 'number')
> +---
> +...
> +-- decimal ----> scalar
> +ok_format_change(11, 'scalar')
> +---
> +...
> +-- decimal --X--> string
> +fail_format_change(11, 'string')
> +---
> +- 'Tuple field 11 type does not match one required by operation: expected string'
> +...
> +-- decimal --X--> integer
> +fail_format_change(11, 'integer')
> +---
> +- 'Tuple field 11 type does not match one required by operation: expected integer'
> +...
> +-- decimal --X--> unsigned
> +fail_format_change(11, 'unsigned')
> +---
> +- 'Tuple field 11 type does not match one required by operation: expected unsigned'
> +...
> +-- decimal --X--> map
> +fail_format_change(11, 'map')
> +---
> +- 'Tuple field 11 type does not match one required by operation: expected map'
> +...
> +-- decimal --X--> array
> +fail_format_change(11, 'array')
> +---
> +- 'Tuple field 11 type does not match one required by operation: expected array'
> +...
> s:drop()
> ---
> ...
> diff --git a/test/engine/ddl.test.lua b/test/engine/ddl.test.lua
> index d15bf1f58..e761966d7 100644
> --- a/test/engine/ddl.test.lua
> +++ b/test/engine/ddl.test.lua
> @@ -355,6 +355,8 @@ s:drop()
> -- gh-2800: space formats checking is broken.
> --
>
> +decimal = require('decimal')
> +
> -- Ensure that vinyl correctly process field count change.
> s = box.schema.space.create('test', {engine = engine, field_count = 2})
> pk = s:create_index('pk')
> @@ -376,9 +378,11 @@ format[7] = {name = 'field7', type = 'boolean'}
> format[8] = {name = 'field8', type = 'scalar'}
> format[9] = {name = 'field9', type = 'array'}
> format[10] = {name = 'field10', type = 'map'}
> +format[11] = {name = 'field11', type = 'decimal'}
> +
> s = box.schema.space.create('test', {engine = engine, format = format})
> pk = s:create_index('pk')
> -t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}}
> +t = s:replace{1, {2}, 3, '4', 5.5, -6, true, -8, {9, 9}, {val = 10}, decimal.new(-11.11)}
>
> inspector:cmd("setopt delimiter ';'")
> function fail_format_change(fieldno, new_type)
> @@ -415,6 +419,8 @@ ok_format_change(3, 'integer')
> ok_format_change(3, 'scalar')
> -- unsigned --X--> map
> fail_format_change(3, 'map')
> +-- unsigned --X--> decimal
> +fail_format_change(3, 'decimal')
>
> -- string -----> any
> ok_format_change(4, 'any')
> @@ -422,6 +428,8 @@ ok_format_change(4, 'any')
> ok_format_change(4, 'scalar')
> -- string --X--> boolean
> fail_format_change(4, 'boolean')
> +-- string --X--> decimal
> +fail_format_change(4, 'decimal')
>
> -- number -----> any
> ok_format_change(5, 'any')
> @@ -429,6 +437,8 @@ ok_format_change(5, 'any')
> ok_format_change(5, 'scalar')
> -- number --X--> integer
> fail_format_change(5, 'integer')
> +-- number --X--> decimal
> +fail_format_change(5, 'decimal')
>
> -- integer -----> any
> ok_format_change(6, 'any')
> @@ -438,6 +448,8 @@ ok_format_change(6, 'number')
> ok_format_change(6, 'scalar')
> -- integer --X--> unsigned
> fail_format_change(6, 'unsigned')
> +-- integer --X--> decimal
> +fail_format_change(6, 'decimal')
>
> -- boolean -----> any
> ok_format_change(7, 'any')
> @@ -445,22 +457,46 @@ ok_format_change(7, 'any')
> ok_format_change(7, 'scalar')
> -- boolean --X--> string
> fail_format_change(7, 'string')
> +-- boolead --X--> decimal
> +fail_format_change(7, 'decimal')
>
> -- scalar -----> any
> ok_format_change(8, 'any')
> -- scalar --X--> unsigned
> fail_format_change(8, 'unsigned')
> +-- scalar --X--> decimal
> +fail_format_change(8, 'decimal')
>
> -- array -----> any
> ok_format_change(9, 'any')
> -- array --X--> scalar
> fail_format_change(9, 'scalar')
> +-- arary --X--> decimal
> +fail_format_change(9, 'decimal')
>
> -- map -----> any
> ok_format_change(10, 'any')
> -- map --X--> scalar
> fail_format_change(10, 'scalar')
> -
> +-- map --X--> decimal
> +fail_format_change(10, 'decimal')
> +
> +-- decimal ----> any
> +ok_format_change(11, 'any')
> +-- decimal ----> number
> +ok_format_change(11, 'number')
> +-- decimal ----> scalar
> +ok_format_change(11, 'scalar')
> +-- decimal --X--> string
> +fail_format_change(11, 'string')
> +-- decimal --X--> integer
> +fail_format_change(11, 'integer')
> +-- decimal --X--> unsigned
> +fail_format_change(11, 'unsigned')
> +-- decimal --X--> map
> +fail_format_change(11, 'map')
> +-- decimal --X--> array
> +fail_format_change(11, 'array')
> s:drop()
>
> -- Check new fields adding.
> diff --git a/test/engine/decimal.result b/test/engine/decimal.result
> index f8888d7c9..4ced88c24 100644
> --- a/test/engine/decimal.result
> +++ b/test/engine/decimal.result
> @@ -12,6 +12,9 @@ engine = test_run:get_cfg('engine')
> decimal = require('decimal')
> | ---
> | ...
> +ffi = require('ffi')
> + | ---
> + | ...
>
> _ = box.schema.space.create('test', {engine=engine})
> | ---
> @@ -76,6 +79,278 @@ box.space.test:select{}
> | - [3, 3.3]
> | - [4, 1234567890123456789.9876543210987654321, 1.2345]
> | ...
> +
> +box.space.test:drop()
> + | ---
> + | ...
> +
> +-- check decimal indexes
> +_ = box.schema.space.create('test', {engine=engine})
> + | ---
> + | ...
> +_ = box.space.test:create_index('pk', {parts={1,'decimal'}})
> + | ---
> + | ...
> +
> +test_run:cmd('setopt delimiter ";"')
> + | ---
> + | - true
> + | ...
> +for i = 0,16 do
> + box.space.test:insert{decimal.new((i-8)/4)}
> +end;
> + | ---
> + | ...
> +test_run:cmd('setopt delimiter ""');
> + | ---
> + | - true
> + | ...
> +
> +box.space.test:select{}
> + | ---
> + | - - [-2]
> + | - [-1.75]
> + | - [-1.5]
> + | - [-1.25]
> + | - [-1]
> + | - [-0.75]
> + | - [-0.5]
> + | - [-0.25]
> + | - [0]
> + | - [0.25]
> + | - [0.5]
> + | - [0.75]
> + | - [1]
> + | - [1.25]
> + | - [1.5]
> + | - [1.75]
> + | - [2]
> + | ...
> +
> +-- check invalid values
> +box.space.test:insert{1.23}
> + | ---
> + | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
> + | ...
> +box.space.test:insert{'str'}
> + | ---
> + | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
> + | ...
> +box.space.test:insert{ffi.new('uint64_t', 0)}
> + | ---
> + | - error: 'Tuple field 1 type does not match one required by operation: expected decimal'
> + | ...
> +-- check duplicates
> +box.space.test:insert{decimal.new(0)}
> + | ---
> + | - error: Duplicate key exists in unique index 'pk' in space 'test'
> + | ...
> +
> +box.space.test.index.pk:drop()
> + | ---
> + | ...
> +
> +_ = box.space.test:create_index('pk', {parts={1, 'number'}})
> + | ---
> + | ...
> +
> +test_run:cmd('setopt delimiter ";"')
> + | ---
> + | - true
> + | ...
> +for i = 0, 32 do
> + local val = (i - 16) / 8
> + if i % 2 == 1 then val = decimal.new(val) end
> + box.space.test:insert{val}
> +end;
> + | ---
> + | ...
> +test_run:cmd('setopt delimiter ""');
> + | ---
> + | - true
> + | ...
> +
> +box.space.test:select{}
> + | ---
> + | - - [-2]
> + | - [-1.875]
> + | - [-1.75]
> + | - [-1.625]
> + | - [-1.5]
> + | - [-1.375]
> + | - [-1.25]
> + | - [-1.125]
> + | - [-1]
> + | - [-0.875]
> + | - [-0.75]
> + | - [-0.625]
> + | - [-0.5]
> + | - [-0.375]
> + | - [-0.25]
> + | - [-0.125]
> + | - [0]
> + | - [0.125]
> + | - [0.25]
> + | - [0.375]
> + | - [0.5]
> + | - [0.625]
> + | - [0.75]
> + | - [0.875]
> + | - [1]
> + | - [1.125]
> + | - [1.25]
> + | - [1.375]
> + | - [1.5]
> + | - [1.625]
> + | - [1.75]
> + | - [1.875]
> + | - [2]
> + | ...
> +
> +-- check duplicates
> +box.space.test:insert{-2}
> + | ---
> + | - error: Duplicate key exists in unique index 'pk' in space 'test'
> + | ...
> +box.space.test:insert{decimal.new(-2)}
> + | ---
> + | - error: Duplicate key exists in unique index 'pk' in space 'test'
> + | ...
> +box.space.test:insert{decimal.new(-1.875)}
> + | ---
> + | - error: Duplicate key exists in unique index 'pk' in space 'test'
> + | ...
> +box.space.test:insert{-1.875}
> + | ---
> + | - error: Duplicate key exists in unique index 'pk' in space 'test'
> + | ...
> +
> +box.space.test.index.pk:drop()
> + | ---
> + | ...
> +
> +_ = box.space.test:create_index('pk')
> + | ---
> + | ...
> +test_run:cmd('setopt delimiter ";"')
> + | ---
> + | - true
> + | ...
> +for i = 1,10 do
> + box.space.test:insert{i, decimal.new(i/10)}
> +end;
> + | ---
> + | ...
> +test_run:cmd('setopt delimiter ""');
> + | ---
> + | - true
> + | ...
> +
> +-- a bigger test with a secondary index this time.
> +box.space.test:insert{11, 'str'}
> + | ---
> + | - [11, 'str']
> + | ...
> +box.space.test:insert{12, 0.63}
> + | ---
> + | - [12, 0.63]
> + | ...
> +box.space.test:insert{13, 0.57}
> + | ---
> + | - [13, 0.57]
> + | ...
> +box.space.test:insert{14, 0.33}
> + | ---
> + | - [14, 0.33]
> + | ...
> +box.space.test:insert{16, 0.71}
> + | ---
> + | - [16, 0.71]
> + | ...
> +
> +_ = box.space.test:create_index('sk', {parts={2, 'scalar'}})
> + | ---
> + | ...
> +box.space.test.index.sk:select{}
> + | ---
> + | - - [1, 0.1]
> + | - [2, 0.2]
> + | - [3, 0.3]
> + | - [14, 0.33]
> + | - [4, 0.4]
> + | - [5, 0.5]
> + | - [13, 0.57]
> + | - [6, 0.6]
> + | - [12, 0.63]
> + | - [7, 0.7]
> + | - [16, 0.71]
> + | - [8, 0.8]
> + | - [9, 0.9]
> + | - [10, 1]
> + | - [11, 'str']
> + | ...
> +
> +box.space.test:drop()
> + | ---
> + | ...
> +
> +-- check index alter
> +_ = box.schema.space.create('test', {engine=engine})
> + | ---
> + | ...
> +_ = box.space.test:create_index('pk')
> + | ---
> + | ...
> +_ = box.space.test:create_index('sk', {parts={2, 'number'}})
> + | ---
> + | ...
> +box.space.test:insert{1, decimal.new(-2)}
> + | ---
> + | - [1, -2]
> + | ...
> +box.space.test:insert{2, -5}
> + | ---
> + | - [2, -5]
> + | ...
> +-- failure
> +box.space.test.index.sk:alter{parts={2, 'decimal'}}
> + | ---
> + | - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
> + | ...
> +box.space.test:delete{2}
> + | ---
> + | ...
> +box.space.test.index.sk:alter{parts={2, 'decimal'}}
> + | ---
> + | ...
> +box.space.test:insert{3, decimal.new(3)}
> + | ---
> + | - [3, 3]
> + | ...
> +--failure
> +box.space.test:insert{4, 'string'}
> + | ---
> + | - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
> + | ...
> +-- failure
> +box.space.test:insert{2, -5}
> + | ---
> + | - error: 'Tuple field 2 type does not match one required by operation: expected decimal'
> + | ...
> +box.space.test.index.sk:alter{parts={2, 'number'}}
> + | ---
> + | ...
> +box.space.test:insert{2, -5}
> + | ---
> + | - [2, -5]
> + | ...
> +box.space.test.index.sk:select{}
> + | ---
> + | - - [2, -5]
> + | - [1, -2]
> + | - [3, 3]
> + | ...
> +
> box.space.test:drop()
> | ---
> | ...
> diff --git a/test/engine/decimal.test.lua b/test/engine/decimal.test.lua
> index 1b14871b0..4e38650dd 100644
> --- a/test/engine/decimal.test.lua
> +++ b/test/engine/decimal.test.lua
> @@ -3,6 +3,7 @@ test_run = env.new()
> engine = test_run:get_cfg('engine')
>
> decimal = require('decimal')
> +ffi = require('ffi')
>
> _ = box.schema.space.create('test', {engine=engine})
> _ = box.space.test:create_index('pk')
> @@ -18,4 +19,86 @@ a:next(2)
> a:slice(-2)
> box.space.test:replace{3, decimal.new(3.3)}
> box.space.test:select{}
> +
> +box.space.test:drop()
> +
> +-- check decimal indexes
> +_ = box.schema.space.create('test', {engine=engine})
> +_ = box.space.test:create_index('pk', {parts={1,'decimal'}})
> +
> +test_run:cmd('setopt delimiter ";"')
> +for i = 0,16 do
> + box.space.test:insert{decimal.new((i-8)/4)}
> +end;
> +test_run:cmd('setopt delimiter ""');
> +
> +box.space.test:select{}
> +
> +-- check invalid values
> +box.space.test:insert{1.23}
> +box.space.test:insert{'str'}
> +box.space.test:insert{ffi.new('uint64_t', 0)}
> +-- check duplicates
> +box.space.test:insert{decimal.new(0)}
> +
> +box.space.test.index.pk:drop()
> +
> +_ = box.space.test:create_index('pk', {parts={1, 'number'}})
> +
> +test_run:cmd('setopt delimiter ";"')
> +for i = 0, 32 do
> + local val = (i - 16) / 8
> + if i % 2 == 1 then val = decimal.new(val) end
> + box.space.test:insert{val}
> +end;
> +test_run:cmd('setopt delimiter ""');
> +
> +box.space.test:select{}
> +
> +-- check duplicates
> +box.space.test:insert{-2}
> +box.space.test:insert{decimal.new(-2)}
> +box.space.test:insert{decimal.new(-1.875)}
> +box.space.test:insert{-1.875}
> +
> +box.space.test.index.pk:drop()
> +
> +_ = box.space.test:create_index('pk')
> +test_run:cmd('setopt delimiter ";"')
> +for i = 1,10 do
> + box.space.test:insert{i, decimal.new(i/10)}
> +end;
> +test_run:cmd('setopt delimiter ""');
> +
> +-- a bigger test with a secondary index this time.
> +box.space.test:insert{11, 'str'}
> +box.space.test:insert{12, 0.63}
> +box.space.test:insert{13, 0.57}
> +box.space.test:insert{14, 0.33}
> +box.space.test:insert{16, 0.71}
> +
> +_ = box.space.test:create_index('sk', {parts={2, 'scalar'}})
> +box.space.test.index.sk:select{}
> +
> +box.space.test:drop()
> +
> +-- check index alter
> +_ = box.schema.space.create('test', {engine=engine})
> +_ = box.space.test:create_index('pk')
> +_ = box.space.test:create_index('sk', {parts={2, 'number'}})
> +box.space.test:insert{1, decimal.new(-2)}
> +box.space.test:insert{2, -5}
> +-- failure
> +box.space.test.index.sk:alter{parts={2, 'decimal'}}
> +box.space.test:delete{2}
> +box.space.test.index.sk:alter{parts={2, 'decimal'}}
> +box.space.test:insert{3, decimal.new(3)}
> +--failure
> +box.space.test:insert{4, 'string'}
> +-- failure
> +box.space.test:insert{2, -5}
> +box.space.test.index.sk:alter{parts={2, 'number'}}
> +box.space.test:insert{2, -5}
> +box.space.test.index.sk:select{}
> +
> box.space.test:drop()
> --
> 2.20.1 (Apple Git-117)
>
^ permalink raw reply [flat|nested] 35+ messages in thread* Re: [PATCH v2 8/8] decimal: allow to index decimals
2019-08-08 11:55 ` [PATCH v2 8/8] decimal: allow to index decimals Serge Petrenko
2019-08-08 13:42 ` Serge Petrenko
@ 2019-08-12 21:41 ` Konstantin Osipov
1 sibling, 0 replies; 35+ messages in thread
From: Konstantin Osipov @ 2019-08-12 21:41 UTC (permalink / raw)
To: Serge Petrenko; +Cc: vdavydov.dev, tarantool-patches
* Serge Petrenko <sergepetrenko@tarantool.org> [19/08/08 15:01]:
This patch is nice. Could you also add iterator tests
(GT/LT/GE/LE/EQ/REQ)?
--
Konstantin Osipov, Moscow, Russia
^ permalink raw reply [flat|nested] 35+ messages in thread