Hi, Sergey!
Thanks for the comments, here is the new commit message:
=======================================================
luajit-gdb: support dualnum mode
 
luajit-gdb.py displays integers in LJ_DUALNUM mode as nan-s. The
dumper function produces output thinking of any input value as of a
double. However, in DUALNUM mode, integers and doubles are stored
differently, so the `itype` of a float number must be less than
`LJ_TISNUM`, and the `itype` of an integer must be `LJ_TISNUM`. With
this fact in mind, we can easily differentiate one from another.
 
Closes tarantool/tarantool#6224
=======================================================
 
And here is the diff:
=======================================================
diff --git a/src/luajit-gdb.py b/src/luajit-gdb.py
index 5f79c277..d4882dd7 100644
--- a/src/luajit-gdb.py
+++ b/src/luajit-gdb.py
@@ -238,8 +238,11 @@ def jit_state(g):
         0x15: 'ERR',
     }.get(int(J(g)['state']), 'INVALID')
+def tvisint(o):
+    return LJ_DUALNUM and itype(o) == LJ_TISNUM
+
 def tvisnumber(o):
-    return itype(o) <= (0xfffeffff if LJ_64 and not LJ_GC64 else LJ_T['NUMX'])
+    return itype(o) <= LJ_TISNUM
 def tvislightud(o):
     if LJ_64 and not LJ_GC64:
@@ -343,9 +346,8 @@ def dump_lj_tudata(tv):
     return 'userdata @ {}'.format(strx64(gcval(tv['gcr'])))
 def dump_lj_tnumx(tv):
-    if itype(tv) == (0xfffeffff if LJ_64 and not LJ_GC64 else LJ_T['NUMX']):
-        integer = cast('int32_t', cast('uint64_t', cast('void*', tv['n'])) & 0xFFFFFFFF)
-        return 'number {}'.format(integer)
+    if tvisint(tv):
+        return 'number {}'.format(cast('int32_t', tv['i']))
     else:
         return 'number {}'.format(cast('double', tv['n']))
@@ -687,7 +689,7 @@ The command requires no args and dumps current GC stats:
         ))
 def init(commands):
-    global LJ_64, LJ_GC64, LJ_FR2, PADDING
+    global LJ_64, LJ_GC64, LJ_DUALNUM, LJ_TISNUM, LJ_FR2, PADDING
     # XXX Fragile: though connecting the callback looks like a crap but it
     # respects both Python 2 and Python 3 (see #4828).
@@ -728,6 +730,7 @@ def init(commands):
     try:
         LJ_64 = str(gdb.parse_and_eval('IRT_PTR')) == 'IRT_P64'
         LJ_FR2 = LJ_GC64 = str(gdb.parse_and_eval('IRT_PGC')) == 'IRT_P64'
+        LJ_DUALNUM = lookup('lj_lib_checknumber') is not None
     except:
         gdb.write('luajit-gdb.py failed to load: '
                   'no debugging symbols found for libluajit\n')
@@ -737,6 +740,7 @@ def init(commands):
         command(name)
     PADDING = ' ' * len(':' + hex((1 << (47 if LJ_GC64 else 32)) - 1))
+    LJ_TISNUM = 0xfffeffff if LJ_64 and not LJ_GC64 else LJ_T['NUMX']
     gdb.write('luajit-gdb.py is successfully loaded\n')
=======================================================
  
>Среда, 11 августа 2021, 11:28 +03:00 от Sergey Kaplun <skaplun@tarantool.org>:
> 
>Hi, Maxim!
>
>Thanks for the patch!
>
>Please consider my comments below.
>
>Side note: First of all, I'm very disappointed, that there this patch
>[1] isn't merged (in any form) into gdb. Those work with the expanding
>of macros is very helpful...
>
>On 31.07.21, Maxim Kokryashkin wrote:
>> For x86/x64 LJ_DUALNUM mode is disabled. But for some other arches
>
>Nit: It can be enabled by corresponding configuration options for
>x86/x64, too, IINM. So I suggest to drop the first and the second
>sentence.
>
>> (arm or arm64, for example) it is enabled by default. luajit-gdb.py
>> displays integers in LJ_DUALNUM mode as nan-s.
>>
>
>Nit: This paragraph may be joined to the next after suggesting changes.
>
>> As it turned out, luajit-gdb detects those integers as integers, but
>> there was a problem with the dumper function itself.
>
>Nit: The next sentence is about the problem with the dumping function, so
>I suggest to drop this opening sentence:)
>Feel free to ignore.
>
>> The dumper
>> function produces output thinking of any input value as of a double.
>> However, in DUALNUM mode, integers and floats are stored differently,
>
>Typo: s/floats/doubles/
>Here and below.
>
>> so the `itype` of a float number must be less than `LJ_TISNUM`, and the
>> `itype` of an integer must be `LJ_TISNUM`. With this fact in mind, we
>> can easily differentiate one from another.
>>
>> But in any mode, lua numbers are stored as doubles on the C side, so it
>
>Typo: s/lua/Lua/
>
>Do you mean LuaJIT here? Because this is not true for LuaJIT. See
><lj_obj.h> for details.
>
>> takes an ugly cast chain on the Python side to perform the some sort of
>> the `reinterpret_cast` because the gdb module doesn't have any
>> mechanism to get the address of a symbol.
>
>This sentence isn't clear to me. What symbol do you mean?
>
>>
>> Closes tarantool/tarantool#6224
>
>Side note: You can say that it really closes the issue, because we use
>luajit-gdb from this fork. OTOH, maybe one uses it as a part of
>third_party from Tarantool repo.
>"Closes" is good to me, but I'm not sure what is idiomatically correct.
>
>> ---
>> Github branch:  https://github.com/tarantool/luajit/tree/fckxorg/gh-6224-support-dulanum
>> Issue:  https://github.com/tarantool/tarantool/issues/6224
>> For more info, see line 273 in lj_obj.h
>>
>> src/luajit-gdb.py | 6 +++++-
>> 1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/luajit-gdb.py b/src/luajit-gdb.py
>> index c50405ad..5f79c277 100644
>> --- a/src/luajit-gdb.py
>> +++ b/src/luajit-gdb.py
>> @@ -343,7 +343,11 @@ def dump_lj_tudata(tv):
>> return 'userdata @ {}'.format(strx64(gcval(tv['gcr'])))
>>
>> def dump_lj_tnumx(tv):
>> - return 'number {}'.format(cast('double', tv['n']))
>> + if itype(tv) == (0xfffeffff if LJ_64 and not LJ_GC64 else LJ_T['NUMX']):
>
>This is true only in LJ_DUALNUM mode. So I suggest we can add another
>one global constant for this: LJ_DUALNUM. We can set it via hack with
>lookuping symbol `lj_lib_checknumber` -- it is compiled only for
>LJ_DUALNUM mode for now. So, if a result of lookup() isn't None we are
>in LJ_DUALNUM mode.
>
>Also, I suggest to create another global constant for LJ_TISNUM, like it
>is done for PADDING constant.
>
>I suppose we want to do something similar to tvisint() macro for this check:
>| LJ_DUALNUM && itype(tv) == LJ_TISNUM
>
>> + integer = cast('int32_t', cast('uint64_t', cast('void*', tv['n'])) & 0xFFFFFFFF)
>
>I don't get this cast to uint64_t and mask. Why can't we just take
>`(int32_t)(o)->i` value, like it is done for `intV()` macro?
>
>> + return 'number {}'.format(integer)
>
>Nit: I suppose, it is better to highlight the fact that TValue stores
>integer here with corresponding return. It may be helpful for debugging
>some issues, related to storing type, I suppose.
>
>But I'm not so sure at this point. Wait for Igor's opinion.
>
>> + else:
>> + return 'number {}'.format(cast('double', tv['n']))
>>
>> def dump_lj_invalid(tv):
>> return 'not valid type @ {}'.format(strx64(gcval(tv['gcr'])))
>> --
>> 2.32.0
>>
>
>[1]:  https://sourceware.org/legacy-ml/gdb-patches/2011-08/msg00441.html
>
>--
>Best regards,
>Sergey Kaplun