[Tarantool-patches] [PATCH] sql: fix number and boolean sorting rules

[Nikita Pettik]

On 24 Apr 16:03, Roman Khabibov wrote:
> Hi! Thanks for the review.
> 
> > On Apr 21, 2020, at 02:57, Nikita Pettik <korablev at tarantool.org> wrote:
> > 
> > On 20 Apr 23:28, Roman Khabibov wrote:

Pushed to master and 2.3 with fixes attached below. Changelogs are
not updated due to the lack of drafts on github. Branch has been dropped.

diff --git a/src/box/sql/vdbeaux.c b/src/box/sql/vdbeaux.c
index 310cec3ed..71f694b20 100644
--- a/src/box/sql/vdbeaux.c
+++ b/src/box/sql/vdbeaux.c
@@ -3053,8 +3053,9 @@ sqlVdbeCompareMsgpack(const char **key1,
                        } else if ((pKey2->flags & MEM_Real) != 0) {
                                rc = double_compare_uint64(pKey2->u.r,
                                                           mem1.u.u, -1);
-                       } else if ((pKey2->flags & MEM_Null) != 0 ||
-                                    (pKey2->flags & MEM_Bool) != 0) {
+                       } else if ((pKey2->flags & MEM_Null) != 0) {
+                               rc = 1;
+                       } else if ((pKey2->flags & MEM_Bool) != 0) {
                                rc = 1;
                        } else {
                                rc = -1;
@@ -3074,8 +3075,9 @@ sqlVdbeCompareMsgpack(const char **key1,
                        } else if (pKey2->flags & MEM_Real) {
                                rc = double_compare_uint64(-pKey2->u.r,
                                                           -mem1.u.i, 1);
-                       } else if ((pKey2->flags & MEM_Null) != 0 ||
-                                    (pKey2->flags & MEM_Bool) != 0) {
+                       } else if ((pKey2->flags & MEM_Null) != 0) {
+                               rc = 1;
+                       } else if ((pKey2->flags & MEM_Bool) != 0) {
                                rc = 1;
                        } else {
                                rc = -1;
@@ -3101,9 +3103,12 @@ sqlVdbeCompareMsgpack(const char **key1,
                                } else if (mem1.u.r > pKey2->u.r) {
                                        rc = +1;
                                }
+                       } else if ((pKey2->flags & MEM_Null) != 0) {
+                               rc = 1;
+                       } else if ((pKey2->flags & MEM_Bool) != 0) {
+                               rc = 1;
                        } else {
-                               rc = (pKey2->flags & MEM_Null) != 0 ||
-                                    (pKey2->flags & MEM_Bool) != 0 ? 1 : -1;
+                               rc = -1;
                        }
                        break;