[Tarantool-patches] [RFC v3] rfc: luajit metrics
Sergey Kaplun
skaplun at tarantool.org
Sun Sep 20 20:13:39 MSK 2020
Part of #5187
---
This patch adds RFC to LuaJIT metrics interfaces. Nevertheless name
`misc` for builtin library is not good and should be discussed, because
tons of user modules can use that name for their own libraries.
Branch: https://github.com/tarantool/tarantool/tree/skaplun/5187-luajit-metrics
Issue: https://github.com/tarantool/tarantool/issues/5187
Changes in v2:
- Fixed typos
- Made comments more verbose
- Avoided flushing any of metrics after each call of luaM_metrics()
Changes in v3:
- Added colors count metrics description
- Added description about how metrics are collected
- Added benchmarks
diff --git a/doc/rfc/5187-luajit-metrics.md b/doc/rfc/5187-luajit-metrics.md
index 4ed066cab..b3839c590 100644
--- a/doc/rfc/5187-luajit-metrics.md
+++ b/doc/rfc/5187-luajit-metrics.md
@@ -48,13 +48,20 @@ struct luam_Metrics {
size_t strhash_miss;
/* Amount of allocated string objects. */
- size_t strnum;
+ size_t gc_strnum;
/* Amount of allocated table objects. */
- size_t tabnum;
+ size_t gc_tabnum;
/* Amount of allocated udata objects. */
- size_t udatanum;
+ size_t gc_udatanum;
/* Amount of allocated cdata objects. */
- size_t cdatanum;
+ size_t gc_cdatanum;
+
+ /* Amount of white objects. */
+ size_t gc_whitenum;
+ /* Amount of gray objects. */
+ size_t gc_graynum;
+ /* Amount of black objects. */
+ size_t gc_blacknum;
/* Memory currently allocated. */
size_t gc_total;
@@ -71,8 +78,10 @@ struct luam_Metrics {
size_t gc_steps_sweep;
size_t gc_steps_finalize;
- /* Overall number of snap restores (amount of guard assertions
- ** leading to stopping trace executions)
+ /*
+ ** Overall number of snap restores (amount of guard assertions
+ ** leading to stopping trace executions and trace exits,
+ ** that are not stitching with other traces).
*/
size_t jit_snap_restore;
/* Overall number of abort traces. */
@@ -84,6 +93,32 @@ struct luam_Metrics {
};
```
+Couple of words about how metrics are collected:
+- `strhash_*` -- whenever existing string is returned after attemption to
+ create new string there is incremented `strhash_hit` counter, if new string
+ created then `strhash_miss` is incremented instead.
+- `gc_*num`, `jit_trace_num` -- corresponding counter incremented whenever new
+ object is allocated. When object become garbage collected its counter is
+ decremented.
+- `gc_whitenum`, `gc_graynum`, `gc_blacknum` -- in so far as all objects are
+ created as the current white, `gc_whitenum` is incremented at any object
+ creation. Whenever color of object changes counter for old color is
+ decremented and counter for new color is incremented instead.
+ *NB*: after full cycle of Garbage Collector there are only white objects.
+- `gc_total`, `gc_allocated`, `gc_freed` -- any time when allocation function
+ is called `gc_allocated` and/or `gc_freed` is increased and `gc_total`
+ increase when memory is allocated or reallocated, decrease when memory is
+ freed.
+- `gc_steps_*` -- corresponding counter increments whenever Garbage Collector
+ starts to execute 1 step of garbage collection.
+- `jit_snap_restore` -- whenever JIT machine exits from the trace and restores
+ interpreter state `jit_snap_restore` counter is incremented.
+- `jit_trace_abort` -- whenever JIT compiler can't record the trace in case NYI
+ BC this counter is incremented.
+- `jit_mcode_size` -- whenever new MCode area is allocated `jit_mcode_size` is
+ increased at corresponding size in bytes. Sets to 0 when all mcode area is
+ freed.
+
All metrics are collected throughout the platform uptime. These metrics
increase monotonically and can overflow:
- `strhash_hit`
@@ -113,24 +148,71 @@ Tarantool 2.5.0-267-gbf047ad44
type 'help' for interactive help
tarantool> misc.getmetrics()
---
-- tabnum: 1812
- gc_total: 1369927
- strnum: 5767
- jit_trace_num: 0
- cdatanum: 89
- jit_mcode_size: 0
- udatanum: 17
- jit_snap_restore: 0
- gc_freed: 2239391
- strhash_hit: 53759
- gc_steps_finalize: 0
- gc_allocated: 3609318
+- gc_graynum: 4443
+ strhash_hit: 53965
gc_steps_atomic: 6
- gc_steps_sweep: 296
+ strhash_miss: 6879
gc_steps_sweepstring: 17920
+ gc_strnum: 5759
+ gc_tabnum: 1813
+ gc_cdatanum: 89
+ jit_snap_restore: 0
+ gc_total: 1370836
+ gc_udatanum: 17
+ gc_steps_finalize: 0
+ gc_allocated: 3616689
+ jit_trace_num: 0
+ gc_whitenum: 3460
+ jit_mcode_size: 0
+ gc_steps_sweep: 297
jit_trace_abort: 0
- strhash_miss: 6874
- gc_steps_propagate: 10106
+ gc_freed: 2245853
gc_steps_pause: 7
+ gc_steps_propagate: 10171
+ gc_blacknum: 3979
...
```
+
+## Benchmarks
+
+Benchmarks was taken from repo:
+[LuaJIT-test-cleanup](https://github.com/LuaJIT/LuaJIT-test-cleanup).
+
+Example of usage:
+```
+/usr/bin/time -f"array3d %U" ./luajit $BENCH_DIR/array3d.lua 300 >/dev/null
+```
+Taking into account the measurement error ~ 2%, it can be said that there is no
+difference in the performance.
+
+Benchmark results after and before patch (less is better):
+```
+ Benchmark | AFTER (s) | BEFORE (s)
+---------------+-----------+-----------
+array3d | 0.21 | 0.20
+binary-trees | 3.34 | 3.24
+chameneos | 2.95 | 2.99
+coroutine-ring | 1.02 | 1.02
+euler14-bit | 1.04 | 1.05
+fannkuch | 6.99 | 6.81
+fasta | 8.28 | 8.28
+life | 0.48 | 0.46
+mandelbrot | 2.66 | 2.68
+mandelbrot-bit | 2.01 | 1.97
+md5 | 1.59 | 1.54
+nbody | 1.36 | 1.56
+nsieve | 2.11 | 2.06
+nsieve-bit | 1.54 | 1.50
+nsieve-bit-fp | 4.51 | 4.60
+partialsums | 0.58 | 0.55
+pidigits-nogmp | 3.48 | 3.46
+ray | 1.62 | 1.63
+recursive-ack | 0.19 | 0.20
+recursive-fib | 1.64 | 1.67
+scimark-fft | 5.84 | 5.86
+scimark-lu | 3.33 | 3.64
+scimark-sor | 2.34 | 2.34
+scimark-sparse | 4.99 | 4.93
+series | 0.95 | 0.94
+spectral-norm | 0.95 | 0.97
+```
More information about the Tarantool-patches
mailing list