From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp53.i.mail.ru (smtp53.i.mail.ru [94.100.177.113]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id E31F344643A for ; Sun, 20 Sep 2020 20:14:02 +0300 (MSK) From: Sergey Kaplun Date: Sun, 20 Sep 2020 20:13:39 +0300 Message-Id: <20200920171339.18866-1-skaplun@tarantool.org> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [Tarantool-patches] [RFC v3] rfc: luajit metrics List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Igor Munkin , Sergey Ostanevich Cc: tarantool-patches@dev.tarantool.org Part of #5187 --- This patch adds RFC to LuaJIT metrics interfaces. Nevertheless name `misc` for builtin library is not good and should be discussed, because tons of user modules can use that name for their own libraries. Branch: https://github.com/tarantool/tarantool/tree/skaplun/5187-luajit-metrics Issue: https://github.com/tarantool/tarantool/issues/5187 Changes in v2: - Fixed typos - Made comments more verbose - Avoided flushing any of metrics after each call of luaM_metrics() Changes in v3: - Added colors count metrics description - Added description about how metrics are collected - Added benchmarks diff --git a/doc/rfc/5187-luajit-metrics.md b/doc/rfc/5187-luajit-metrics.md index 4ed066cab..b3839c590 100644 --- a/doc/rfc/5187-luajit-metrics.md +++ b/doc/rfc/5187-luajit-metrics.md @@ -48,13 +48,20 @@ struct luam_Metrics { size_t strhash_miss; /* Amount of allocated string objects. */ - size_t strnum; + size_t gc_strnum; /* Amount of allocated table objects. */ - size_t tabnum; + size_t gc_tabnum; /* Amount of allocated udata objects. */ - size_t udatanum; + size_t gc_udatanum; /* Amount of allocated cdata objects. */ - size_t cdatanum; + size_t gc_cdatanum; + + /* Amount of white objects. */ + size_t gc_whitenum; + /* Amount of gray objects. */ + size_t gc_graynum; + /* Amount of black objects. */ + size_t gc_blacknum; /* Memory currently allocated. */ size_t gc_total; @@ -71,8 +78,10 @@ struct luam_Metrics { size_t gc_steps_sweep; size_t gc_steps_finalize; - /* Overall number of snap restores (amount of guard assertions - ** leading to stopping trace executions) + /* + ** Overall number of snap restores (amount of guard assertions + ** leading to stopping trace executions and trace exits, + ** that are not stitching with other traces). */ size_t jit_snap_restore; /* Overall number of abort traces. */ @@ -84,6 +93,32 @@ struct luam_Metrics { }; ``` +Couple of words about how metrics are collected: +- `strhash_*` -- whenever existing string is returned after attemption to + create new string there is incremented `strhash_hit` counter, if new string + created then `strhash_miss` is incremented instead. +- `gc_*num`, `jit_trace_num` -- corresponding counter incremented whenever new + object is allocated. When object become garbage collected its counter is + decremented. +- `gc_whitenum`, `gc_graynum`, `gc_blacknum` -- in so far as all objects are + created as the current white, `gc_whitenum` is incremented at any object + creation. Whenever color of object changes counter for old color is + decremented and counter for new color is incremented instead. + *NB*: after full cycle of Garbage Collector there are only white objects. +- `gc_total`, `gc_allocated`, `gc_freed` -- any time when allocation function + is called `gc_allocated` and/or `gc_freed` is increased and `gc_total` + increase when memory is allocated or reallocated, decrease when memory is + freed. +- `gc_steps_*` -- corresponding counter increments whenever Garbage Collector + starts to execute 1 step of garbage collection. +- `jit_snap_restore` -- whenever JIT machine exits from the trace and restores + interpreter state `jit_snap_restore` counter is incremented. +- `jit_trace_abort` -- whenever JIT compiler can't record the trace in case NYI + BC this counter is incremented. +- `jit_mcode_size` -- whenever new MCode area is allocated `jit_mcode_size` is + increased at corresponding size in bytes. Sets to 0 when all mcode area is + freed. + All metrics are collected throughout the platform uptime. These metrics increase monotonically and can overflow: - `strhash_hit` @@ -113,24 +148,71 @@ Tarantool 2.5.0-267-gbf047ad44 type 'help' for interactive help tarantool> misc.getmetrics() --- -- tabnum: 1812 - gc_total: 1369927 - strnum: 5767 - jit_trace_num: 0 - cdatanum: 89 - jit_mcode_size: 0 - udatanum: 17 - jit_snap_restore: 0 - gc_freed: 2239391 - strhash_hit: 53759 - gc_steps_finalize: 0 - gc_allocated: 3609318 +- gc_graynum: 4443 + strhash_hit: 53965 gc_steps_atomic: 6 - gc_steps_sweep: 296 + strhash_miss: 6879 gc_steps_sweepstring: 17920 + gc_strnum: 5759 + gc_tabnum: 1813 + gc_cdatanum: 89 + jit_snap_restore: 0 + gc_total: 1370836 + gc_udatanum: 17 + gc_steps_finalize: 0 + gc_allocated: 3616689 + jit_trace_num: 0 + gc_whitenum: 3460 + jit_mcode_size: 0 + gc_steps_sweep: 297 jit_trace_abort: 0 - strhash_miss: 6874 - gc_steps_propagate: 10106 + gc_freed: 2245853 gc_steps_pause: 7 + gc_steps_propagate: 10171 + gc_blacknum: 3979 ... ``` + +## Benchmarks + +Benchmarks was taken from repo: +[LuaJIT-test-cleanup](https://github.com/LuaJIT/LuaJIT-test-cleanup). + +Example of usage: +``` +/usr/bin/time -f"array3d %U" ./luajit $BENCH_DIR/array3d.lua 300 >/dev/null +``` +Taking into account the measurement error ~ 2%, it can be said that there is no +difference in the performance. + +Benchmark results after and before patch (less is better): +``` + Benchmark | AFTER (s) | BEFORE (s) +---------------+-----------+----------- +array3d | 0.21 | 0.20 +binary-trees | 3.34 | 3.24 +chameneos | 2.95 | 2.99 +coroutine-ring | 1.02 | 1.02 +euler14-bit | 1.04 | 1.05 +fannkuch | 6.99 | 6.81 +fasta | 8.28 | 8.28 +life | 0.48 | 0.46 +mandelbrot | 2.66 | 2.68 +mandelbrot-bit | 2.01 | 1.97 +md5 | 1.59 | 1.54 +nbody | 1.36 | 1.56 +nsieve | 2.11 | 2.06 +nsieve-bit | 1.54 | 1.50 +nsieve-bit-fp | 4.51 | 4.60 +partialsums | 0.58 | 0.55 +pidigits-nogmp | 3.48 | 3.46 +ray | 1.62 | 1.63 +recursive-ack | 0.19 | 0.20 +recursive-fib | 1.64 | 1.67 +scimark-fft | 5.84 | 5.86 +scimark-lu | 3.33 | 3.64 +scimark-sor | 2.34 | 2.34 +scimark-sparse | 4.99 | 4.93 +series | 0.95 | 0.94 +spectral-norm | 0.95 | 0.97 +```