[Tarantool-patches] [RFC v3] rfc: luajit metrics

[Sergey Kaplun <skaplun@tarantool.org>]

Part of #5187
---

This patch adds RFC to LuaJIT metrics interfaces. Nevertheless name
`misc` for builtin library is not good and should be discussed, because
tons of user modules can use that name for their own libraries.

Branch: https://github.com/tarantool/tarantool/tree/skaplun/5187-luajit-metrics
Issue: https://github.com/tarantool/tarantool/issues/5187

Changes in v2:
- Fixed typos
- Made comments more verbose
- Avoided flushing any of metrics after each call of luaM_metrics()
Changes in v3:
- Added colors count metrics description
- Added description about how metrics are collected
- Added benchmarks

diff --git a/doc/rfc/5187-luajit-metrics.md b/doc/rfc/5187-luajit-metrics.md
index 4ed066cab..b3839c590 100644
--- a/doc/rfc/5187-luajit-metrics.md
+++ b/doc/rfc/5187-luajit-metrics.md
@@ -48,13 +48,20 @@ struct luam_Metrics {
   size_t strhash_miss;
 
   /* Amount of allocated string objects. */
-  size_t strnum;
+  size_t gc_strnum;
   /* Amount of allocated table objects. */
-  size_t tabnum;
+  size_t gc_tabnum;
   /* Amount of allocated udata objects. */
-  size_t udatanum;
+  size_t gc_udatanum;
   /* Amount of allocated cdata objects. */
-  size_t cdatanum;
+  size_t gc_cdatanum;
+
+  /* Amount of white objects. */
+  size_t gc_whitenum;
+  /* Amount of gray objects. */
+  size_t gc_graynum;
+  /* Amount of black objects. */
+  size_t gc_blacknum;
 
   /* Memory currently allocated. */
   size_t gc_total;
@@ -71,8 +78,10 @@ struct luam_Metrics {
   size_t gc_steps_sweep;
   size_t gc_steps_finalize;
 
-  /* Overall number of snap restores (amount of guard assertions
-  ** leading to stopping trace executions)
+  /*
+  ** Overall number of snap restores (amount of guard assertions
+  ** leading to stopping trace executions and trace exits,
+  ** that are not stitching with other traces).
   */
   size_t jit_snap_restore;
   /* Overall number of abort traces. */
@@ -84,6 +93,32 @@ struct luam_Metrics {
 };
 ```
 
+Couple of words about how metrics are collected:
+- `strhash_*` -- whenever existing string is returned after attemption to
+  create new string there is incremented `strhash_hit` counter, if new string
+  created then `strhash_miss` is incremented instead.
+- `gc_*num`, `jit_trace_num` -- corresponding counter incremented whenever new
+  object is allocated. When object become garbage collected its counter is
+  decremented.
+- `gc_whitenum`, `gc_graynum`, `gc_blacknum` -- in so far as all objects are
+  created  as the current white, `gc_whitenum` is incremented at any object
+  creation. Whenever  color of object changes counter for old color is
+  decremented and counter for  new color is incremented instead.
+  *NB*: after full cycle of Garbage Collector there are only white objects.
+- `gc_total`, `gc_allocated`, `gc_freed` -- any time when allocation function
+  is called `gc_allocated` and/or `gc_freed` is increased and `gc_total`
+  increase when memory is allocated or reallocated, decrease when memory is
+  freed.
+- `gc_steps_*` -- corresponding counter increments whenever Garbage Collector
+  starts to execute 1 step of garbage collection.
+- `jit_snap_restore` -- whenever JIT machine exits from the trace and restores
+  interpreter state `jit_snap_restore` counter is incremented.
+- `jit_trace_abort` -- whenever JIT compiler can't record the trace in case NYI
+  BC this counter is incremented.
+- `jit_mcode_size` -- whenever new MCode area is allocated `jit_mcode_size` is
+  increased at corresponding size in bytes. Sets to 0 when all mcode area is
+  freed.
+
 All metrics are collected throughout the platform uptime. These metrics
 increase monotonically and can overflow:
   - `strhash_hit`
@@ -113,24 +148,71 @@ Tarantool 2.5.0-267-gbf047ad44
 type 'help' for interactive help
 tarantool> misc.getmetrics()
 ---
-- tabnum: 1812
-  gc_total: 1369927
-  strnum: 5767
-  jit_trace_num: 0
-  cdatanum: 89
-  jit_mcode_size: 0
-  udatanum: 17
-  jit_snap_restore: 0
-  gc_freed: 2239391
-  strhash_hit: 53759
-  gc_steps_finalize: 0
-  gc_allocated: 3609318
+- gc_graynum: 4443
+  strhash_hit: 53965
   gc_steps_atomic: 6
-  gc_steps_sweep: 296
+  strhash_miss: 6879
   gc_steps_sweepstring: 17920
+  gc_strnum: 5759
+  gc_tabnum: 1813
+  gc_cdatanum: 89
+  jit_snap_restore: 0
+  gc_total: 1370836
+  gc_udatanum: 17
+  gc_steps_finalize: 0
+  gc_allocated: 3616689
+  jit_trace_num: 0
+  gc_whitenum: 3460
+  jit_mcode_size: 0
+  gc_steps_sweep: 297
   jit_trace_abort: 0
-  strhash_miss: 6874
-  gc_steps_propagate: 10106
+  gc_freed: 2245853
   gc_steps_pause: 7
+  gc_steps_propagate: 10171
+  gc_blacknum: 3979
 ...
 ```
+
+## Benchmarks
+
+Benchmarks was taken from repo:
+[LuaJIT-test-cleanup](https://github.com/LuaJIT/LuaJIT-test-cleanup).
+
+Example of usage:
+```
+/usr/bin/time -f"array3d %U" ./luajit $BENCH_DIR/array3d.lua  300 >/dev/null
+```
+Taking into account the measurement error ~ 2%, it can be said that there is no
+difference in the performance.
+
+Benchmark results after and before patch (less is better):
+```
+   Benchmark   | AFTER (s) | BEFORE (s)
+---------------+-----------+-----------
+array3d        |   0.21    |   0.20
+binary-trees   |   3.34    |   3.24
+chameneos      |   2.95    |   2.99
+coroutine-ring |   1.02    |   1.02
+euler14-bit    |   1.04    |   1.05
+fannkuch       |   6.99    |   6.81
+fasta          |   8.28    |   8.28
+life           |   0.48    |   0.46
+mandelbrot     |   2.66    |   2.68
+mandelbrot-bit |   2.01    |   1.97
+md5            |   1.59    |   1.54
+nbody          |   1.36    |   1.56
+nsieve         |   2.11    |   2.06
+nsieve-bit     |   1.54    |   1.50
+nsieve-bit-fp  |   4.51    |   4.60
+partialsums    |   0.58    |   0.55
+pidigits-nogmp |   3.48    |   3.46
+ray            |   1.62    |   1.63
+recursive-ack  |   0.19    |   0.20
+recursive-fib  |   1.64    |   1.67
+scimark-fft    |   5.84    |   5.86
+scimark-lu     |   3.33    |   3.64
+scimark-sor    |   2.34    |   2.34
+scimark-sparse |   4.99    |   4.93
+series         |   0.95    |   0.94
+spectral-norm  |   0.95    |   0.97
+```