[Tarantool-patches] [PATCH v2 luajit 0/3] Introduce gdb extension for LuaJIT

[Igor Munkin]

The series provides a gdb extension with commands for inspecting LuaJIT
internals. To use it, just put 'source <path-to-repo>/src/luajit-gdb.py'
in gdb. The extension obliges the one to provide gdbinfo for libluajit,
otherwise loading can fail with the error:

| (gdb) source <path-to-repo>/src/luajit-gdb.py
| luajit-gdb.py failed to load: no debugging symbols found for libluajit

When the loading succeeds the one see the following:

| (gdb) source <path-to-repo>/src/luajit-gdb.py
| lj-arch command initialized
| lj-tv command initialized
| lj-str command initialized
| lj-tab command initialized
| lj-stack command initialized
| lj-state command initialized
| lj-gc command initialized
| luajit-gdb.py is successfully loaded

Below is a description for the set of implemented commands below:

### lj-arch

The command requires no args and dumps values of LJ_64 and LJ_GC64
compile-time flags. These values define the sizes of host and GC
pointers respectively.

NB: Compile-time defines are checked when script is being loaded:
* When LJ_64 is enabled IRT_PTR is an IRT_P64 alias and an IRT_P32 one
  otherwise
* When LJ_64 is enabled IRT_PGC is an IRT_P64 alias and an IRT_P32 one
  otherwise

### lj-tv

The command recieves a <tv address> (TValue address) and dumps the type
and some info related to it.
* LJ_TNIL: nil
* LJ_TFALSE: false
* LJ_TTRUE: true
* LJ_TLIGHTUD: light userdata @ <gcr>
* LJ_TSTR: string <string payload> @ <gcr>
* LJ_TUPVAL: upvalue @ <gcr>
* LJ_TTHREAD: thread @ <gcr>
* LJ_TPROTO: proto @ <gcr>
* LJ_TFUNC: <LFUNC|CFUNC|FFUNC>
  - <LFUNC>: Lua function @ <gcr>, <nupvals> upvalues, <chunk:line>
  - <CFUNC>: C function <mcode address>
  - <FFUNC>: fast function #<ffid>
* LJ_TTRACE: trace <traceno> @ <gcr>
* LJ_TCDATA: cdata @ <gcr>
* LJ_TTAB: table @ <gcr> (asize: <asize>, hmask: <hmask>)
* LJ_TUDATA: userdata @ <gcr>
* LJ_TNUMX: number <numeric payload>

Whether the type of the given address differs from the listed above, then
error message occurs.

### lj-str

The command recieves a <gcr> of the corresponding GCstr object and dumps
the payload, size in bytes and hash.

*Caveat*: Since Python 2 provides no native Unicode support, the payload
is replaced with the corresponding error when decoding fails.

### lj-tab

The command recieves a GCtab address and dumps the table contents:
* Metatable address whether the one is set
* Array part <asize> slots:
  <aslot ptr>: [<index>]: <tv>
* Hash part <hsize> nodes:
  <hnode ptr>: { <tv> } => { <tv> }; next = <next hnode ptr>

### lj-stack

The command recieves a lua_State address and dumps the given Lua
coroutine guest stack:

<slot ptr> [<slot attributes>] <VALUE|FRAME>

* <slot ptr>: guest stack slot address
* <slot attributes>:
  - S: Bottom of the stack (the slot L->stack points to)
  - B: Base of the current guest frame (the slot L->base points to)
  - T: Top of the current guest frame (the slot L->top points to)
  - M: Last slot of the stack (the slot L->maxstack points to)
* <VALUE>: see help lj-tv for more info
* <FRAME>: framelink slot differs from the value slot: it contains info
  related to the function being executed within this guest frame, its
  type and link to the parent guest frame
  [<frame type>] delta=<slots in frame>, <lj-tv for LJ_TFUNC slot>
  - <frame type>:
    + L:  VM performs a call as a result of bytecode execution
    + C:  VM performs a call as a result of lj_vm_call
    + M:  VM performs a call to a metamethod as a result of bytecode
          execution
    + V:  Variable-length frame for storing arguments of a variadic
          function
    + CP: Protected C frame
    + PP: VM performs a call as a result of executinig pcall or xpcall

If L is ommited the main coroutine is used.

### lj-state

The command requires no args and dumps current VM and GC states
* VM state: <INTERP|C|GC|EXIT|RECORD|OPT|ASM|TRACE>
* GC state: <PAUSE|PROPAGATE|ATOMIC|SWEEPSTRING|SWEEP|FINALIZE|LAST>
* JIT state: <IDLE|ACTIVE|RECORD|START|END|ASM|ERR>

### lj-gc

The command requires no args and dumps current GC stats:
* total: <total number of allocated bytes in GC area>
* threshold: <limit when gc step is triggered>
* debt: <how much GC is behind schedule>
* estimate: <estimate of memory actually in use>
* stepmul: <incremental GC step granularity>
* pause: <pause between successive GC cycles>
* sweepstr: <sweep position in string table>
* root: <number of all collectable objects>
* gray: <number of gray objects>
* grayagain: <number of objects for atomic traversal>
* weak: <number of weak tables (to be cleared)>

Signed-off-by: Igor Munkin <imun at tarantool.org>

--

Changes in v2:
* squashed "fixup" commits with the first one
* added loading errors handling

v1: https://lists.tarantool.org/pipermail/tarantool-patches/2020-January/013801.html

Branch: https://github.com/tarantool/luajit/tree/imun/luajit-gdb

Igor Munkin (3):
  gdb: introduce luajit-gdb extension
  gdb: adjust the extension to be used with Python 2
  gdb: enhance the extension loading

 src/luajit-gdb.py | 687 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 687 insertions(+)
 create mode 100644 src/luajit-gdb.py

-- 
2.24.0