Tarantool development patches archive
 help / color / mirror / Atom feed
From: Serge Petrenko <sergepetrenko@tarantool.org>
To: v.shpilevoy@tarantool.org
Cc: tarantool-patches@dev.tarantool.org
Subject: [Tarantool-patches] [PATCH v3 3/3] app/fiber: wait till a full event loop iteration ends
Date: Mon, 18 Nov 2019 19:05:05 +0300	[thread overview]
Message-ID: <f5ef6b1bf389942bc1e405643fec55381ac83d95.1574091776.git.sergepetrenko@tarantool.org> (raw)
In-Reply-To: <cover.1574091776.git.sergepetrenko@tarantool.org>

fiber.top() fills in statistics every event loop iteration,
so if it was just enabled, fiber.top() returns zero in fiber cpu
usage statistics because total time consumed by the main thread was
not yet accounted for.
Same stands for viewing top() results for a freshly created fiber:
its metrics will be zero since it hasn't lived a full ev loop iteration
yet.
Fix this by delaying the test till top() results are meaningful and add
minor refactoring.

Follow-up #2694
---
 test/app/fiber.result   | 38 +++++++++++++++++++++++++++++---------
 test/app/fiber.test.lua | 36 ++++++++++++++++++++++++++++--------
 2 files changed, 57 insertions(+), 17 deletions(-)

diff --git a/test/app/fiber.result b/test/app/fiber.result
index 4a094939f..b767bbb59 100644
--- a/test/app/fiber.result
+++ b/test/app/fiber.result
@@ -1469,6 +1469,12 @@ sum = 0
 fiber.top_enable()
 ---
 ...
+-- Wait till a full event loop iteration passes, so that
+-- top() contains meaningful results. On the ev loop iteration
+-- following fiber.top_enable() results will be zero.
+while fiber.top().cpu["1/sched"].instant == 0 do fiber.yield() end
+---
+...
 a = fiber.top()
 ---
 ...
@@ -1508,33 +1514,47 @@ sum_inst
 ---
 - 100
 ...
--- not exact due to accumulated integer division errors
-sum_avg > 99 and sum_avg < 101 or sum_avg
+-- when a fiber dies, its impact on the thread moving average
+-- persists for a couple of ev loop iterations, but it is no
+-- longer listed in fiber.top(). So sum_avg may way smaller than
+-- 100%. See gh-4625 for details and reenable the test below as
+-- soon as it is implemented.
+-- In rare cases when a fiber dies on the same event loop
+-- iteration as you issue fiber.top(), sum_inst will also be
+-- smaller than 100%, but it is so rare I don't even want to
+-- disable the test above.
+sum_avg <= 100 or sum_avg
 ---
 - true
 ...
+-- not exact due to accumulated integer division errors
+--sum_avg > 99 and sum_avg <= 100 or sum_avg
 tbl = nil
 ---
 ...
 f = fiber.new(function()\
-    for i = 1,1000 do end\
-    fiber.yield()\
-    tbl = fiber.top().cpu[fiber.self().id()..'/'..fiber.self().name()]\
+    local fiber_key = fiber.self().id()..'/'..fiber.self().name()\
+    tbl = fiber.top().cpu[fiber_key]\
+    while tbl.time == 0 do\
+        for i = 1,1000 do end\
+        fiber.yield()\
+        tbl = fiber.top().cpu[fiber_key]\
+    end\
 end)
 ---
 ...
-while f:status() ~= 'dead' do fiber.sleep(0.01) end
+while f:status() ~= 'dead' do fiber.yield() end
 ---
 ...
-tbl["average"] > 0
+tbl.average > 0
 ---
 - true
 ...
-tbl["instant"] > 0
+tbl.instant > 0
 ---
 - true
 ...
-tbl["time"] > 0
+tbl.time > 0
 ---
 - true
 ...
diff --git a/test/app/fiber.test.lua b/test/app/fiber.test.lua
index 38b85d554..7f7350acc 100644
--- a/test/app/fiber.test.lua
+++ b/test/app/fiber.test.lua
@@ -634,6 +634,12 @@ sum = 0
 -- gh-2694 fiber.top()
 fiber.top_enable()
 
+
+-- Wait till a full event loop iteration passes, so that
+-- top() contains meaningful results. On the ev loop iteration
+-- following fiber.top_enable() results will be zero.
+while fiber.top().cpu["1/sched"].instant == 0 do fiber.yield() end
+
 a = fiber.top()
 type(a)
 -- scheduler is present in fiber.top()
@@ -653,18 +659,32 @@ for k, v in pairs(a) do\
 end
 
 sum_inst
+-- when a fiber dies, its impact on the thread moving average
+-- persists for a couple of ev loop iterations, but it is no
+-- longer listed in fiber.top(). So sum_avg may way smaller than
+-- 100%. See gh-4625 for details and reenable the test below as
+-- soon as it is implemented.
+-- In rare cases when a fiber dies on the same event loop
+-- iteration as you issue fiber.top(), sum_inst will also be
+-- smaller than 100%, but it is so rare I don't even want to
+-- disable the test above.
+sum_avg <= 100 or sum_avg
 -- not exact due to accumulated integer division errors
-sum_avg > 99 and sum_avg < 101 or sum_avg
+--sum_avg > 99 and sum_avg <= 100 or sum_avg
 tbl = nil
 f = fiber.new(function()\
-    for i = 1,1000 do end\
-    fiber.yield()\
-    tbl = fiber.top().cpu[fiber.self().id()..'/'..fiber.self().name()]\
+    local fiber_key = fiber.self().id()..'/'..fiber.self().name()\
+    tbl = fiber.top().cpu[fiber_key]\
+    while tbl.time == 0 do\
+        for i = 1,1000 do end\
+        fiber.yield()\
+        tbl = fiber.top().cpu[fiber_key]\
+    end\
 end)
-while f:status() ~= 'dead' do fiber.sleep(0.01) end
-tbl["average"] > 0
-tbl["instant"] > 0
-tbl["time"] > 0
+while f:status() ~= 'dead' do fiber.yield() end
+tbl.average > 0
+tbl.instant > 0
+tbl.time > 0
 
 fiber.top_disable()
 fiber.top()
-- 
2.21.0 (Apple Git-122)

  parent reply	other threads:[~2019-11-18 16:05 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-18 16:05 [Tarantool-patches] [PATCH v3 0/3] fiber.top(): minor fixup Serge Petrenko
2019-11-18 16:05 ` [Tarantool-patches] [PATCH v3 1/3] fiber.top() refactor clock and cpu time calculation Serge Petrenko
2019-11-18 22:25   ` Vladislav Shpilevoy
2019-11-19  7:14     ` Sergey Petrenko
2019-11-18 16:05 ` [Tarantool-patches] [PATCH v3 2/3] fiber.top(): alter exponential moving average calculation Serge Petrenko
2019-11-18 16:05 ` Serge Petrenko [this message]
2019-11-19 22:57 ` [Tarantool-patches] [PATCH v3 0/3] fiber.top(): minor fixup Vladislav Shpilevoy
2019-11-20  1:44   ` Alexander Turenko
2019-11-20  8:29     ` Serge Petrenko
2019-11-20 12:19       ` Alexander Turenko
2019-11-21 17:07 ` Kirill Yukhin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f5ef6b1bf389942bc1e405643fec55381ac83d95.1574091776.git.sergepetrenko@tarantool.org \
    --to=sergepetrenko@tarantool.org \
    --cc=tarantool-patches@dev.tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH v3 3/3] app/fiber: wait till a full event loop iteration ends' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox