[Tarantool-patches] [PATCH luajit] luajit-gdb: support dualnum mode

[Maxim Kokryashkin via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

For x86/x64 LJ_DUALNUM mode is disabled. But for some other arches
(arm or arm64, for example) it is enabled by default. luajit-gdb.py
displays integers in LJ_DUALNUM mode as nan-s.

As it turned out, luajit-gdb detects those integers as integers, but
there was a problem with the dumper function itself. The dumper
function produces output thinking of any input value as of a double.
However, in DUALNUM mode, integers and floats are stored differently,
so the `itype` of a float number must be less than `LJ_TISNUM`, and the
`itype` of an integer must be `LJ_TISNUM`. With this fact in mind, we
can easily differentiate one from another.

But in any mode, lua numbers are stored as doubles on the C side, so it
takes an ugly cast chain on the Python side to perform the some sort of
the `reinterpret_cast` because the gdb module doesn't have any
mechanism to get the address of a symbol.

Closes tarantool/tarantool#6224
---
Github branch: https://github.com/tarantool/luajit/tree/fckxorg/gh-6224-support-dulanum
Issue: https://github.com/tarantool/tarantool/issues/6224
For more info, see line 273 in lj_obj.h

 src/luajit-gdb.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/src/luajit-gdb.py b/src/luajit-gdb.py
index c50405ad..5f79c277 100644
--- a/src/luajit-gdb.py
+++ b/src/luajit-gdb.py
@@ -343,7 +343,11 @@ def dump_lj_tudata(tv):
     return 'userdata @ {}'.format(strx64(gcval(tv['gcr'])))
 
 def dump_lj_tnumx(tv):
-    return 'number {}'.format(cast('double', tv['n']))
+    if itype(tv) == (0xfffeffff if LJ_64 and not LJ_GC64 else LJ_T['NUMX']):
+        integer = cast('int32_t', cast('uint64_t', cast('void*', tv['n'])) & 0xFFFFFFFF)
+        return 'number {}'.format(integer)
+    else:
+        return 'number {}'.format(cast('double', tv['n']))
 
 def dump_lj_invalid(tv):
     return 'not valid type @ {}'.format(strx64(gcval(tv['gcr'])))
-- 
2.32.0