From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 5F3F02488F for ; Thu, 25 Jul 2019 09:02:56 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id inNa5m6lprIh for ; Thu, 25 Jul 2019 09:02:56 -0400 (EDT) Received: from smtp50.i.mail.ru (smtp50.i.mail.ru [94.100.177.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id A40392488A for ; Thu, 25 Jul 2019 09:02:55 -0400 (EDT) From: =?utf-8?B?0JPQtdC+0YDQs9C40Lkg0JrQuNGA0LjRh9C10L3QutC+?= Subject: [tarantool-patches] Re: [PATCH] Output of fiber.info will contain only non-idle fibers Date: Thu, 25 Jul 2019 16:02:49 +0300 Message-ID: <4500632.28QgJ9nIKZ@localhost> In-Reply-To: <20190725092640.GJ15185@atlas> References: <2274932.tPdvVCckH0@home.lan> <20190725092640.GJ15185@atlas> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart4100605.4X5qbz7Pr1"; micalg="pgp-sha256"; protocol="application/pgp-signature" Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: Konstantin Osipov Cc: tarantool-patches@freelists.org --nextPart4100605.4X5qbz7Pr1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Thursday, July 25, 2019 12:26:40 PM MSK Konstantin Osipov wrote: > * =D0=93=D0=B5=D0=BE=D1=80=D0=B3=D0=B8=D0=B9 =D0=9A=D0=B8=D1=80=D0=B8=D1= =87=D0=B5=D0=BD=D0=BA=D0=BE [19/07/25 10:05]: > > On Tuesday, July 23, 2019 10:56:43 PM MSK Konstantin Osipov wrote: > > > * Maria K [19/07/23 21:01]: > > > > The output used to be too cluttered due to idle ones. > > > >=20 > > > > Closes #4235 > > >=20 > > > @kyukhin, first, please I don't get how does this get scheduled to a > > > milestone? How does this follow triage guidelines? > > >=20 > > > Please don't schedule anything that is not a priority, even if > > > it's a noob issue, since it takes time of everyone involved. > >=20 > > I think anybody is free to send a patch to the public tarantool > > mailing list despite the issue milestone (if they is not bound > > by employee duties). Also it was a 'good first issue' ticket to > > start a candidate on boarding. >=20 > The reason it was a good first issue is that the problem was > wrongly defined in the first place. As defined, it had a trivial > solution. >=20 > The mere fact we have a disagreement suggests the label was > applied incorrectly. >=20 > > > fiber.info() already doesn't show anything from cord->dead list. > > > Fibers which are stuck in a pool are performing application-level > > > code, even if it's a built in pool, so contribute valuable > > > information to fiber.info() output. Besides, it's always easy to > > > filter out any class of fibers with luafun. > >=20 > > It is not easy to filter out such fibers. In the other hand tx fiber po= ol > > is 'a hack' to spare some fiber structures between invocations. So fiber > > pool cached fibers could/should be threatened as dead ones. >=20 > Well, I don't think everyone should be unconditionally deprived of > this data if someone can't write a single-line snippet with luafun. There is no point to unwind such fibers stacks which costs a lot as well as= =20 produces a lot of lua garbage. >=20 > > > Finally, there are other types of pools -- an application-level pool > > > in Lua will have lots of idle fibers in it. > >=20 > > An application level fiber pool uses some user-defined condition with > > exactly- defied meaning and state. And it isn't the same as the tx fiber > > pool. An appplication fiber (in pool or not) is the resource managed by > > user while tx fiber pool is not. And I see no point in seeing an idle > > fiber from tx fiber pool. > I disagree there is no point in seeing these fibers. They take up > fiber stack and contribute to the total list libev events/fibers the > scheduler has to deal with. If you checked the code you could see that such fibers do not consume libev= =20 events and participated only in alive list without any scheduling duties. S= uch=20 fibers use only fiber stack which is already accounted using memory statist= ics. >=20 > > > In other words, this is an partial fix of a raw feature > > > request. > > >=20 > > > Tarantool instrumentation sucks, but it doesn't mean it should be > > > patched by quick hacks here and there. > > >=20 > > > A nice and general solution would be to compress mostly identical > > > fiber.info() entries. But I guess it's not a noob task. > >=20 > > I didn't find your suggestion solution nice and general in case of > > filtering idle fibers out. >=20 > Another way of properly fixing it would be to more > aggressively/carefully This makes fiber pool worthless because fiber pool has only two purposes: t= o=20 limit the amount of fibers and to spare some resources. I would like to rep= eat:=20 a fiber cached in a fiber pool is a dead fiber without any state. > expire such fibers from the pool. > If you look at the current idle > callback implementation there are a few flaws in it: > - there is a standalone idle callback, rather than fiber idle > timeout, and the callback removes no more than 1 fiber per > second > - when the idle callback wakes up a fiber, it doesn't necessarily > die. It looks at the pool->output first, and if there are > messages in the output, works on them. Which is apparently > wrong, because there always messages in the list on a busy > system, this doesn't mean the idle fiber should be the one to > work them off. Your were right only in case when a system has a stable rps what is definit= ely=20 not the common case. And if a system has no idle fibers (in case of stable = rps)=20 there is no reason to have a fiber pool. >=20 > *None* of this would be visible/bothering anybody if this > information was hidden from fiber.info(). And we want to hide fiber pool internals from user by filtering idle (dead,= =20 state-less, unused, resource-less) fibers out. >=20 > So this begs the question: why am I wasting time > discussing/explaining this, how did it suddenly become a priority, > which I asked in the beginning of this thread. It was only your decision to discuss this as you are free to left it out. A= s=20 anybody else is free to submit a patch. --nextPart4100605.4X5qbz7Pr1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEFZT35EtIMRTDS5hJnoTdFFzh6LUFAl05qHkACgkQnoTdFFzh 6LX0lQf/bvHaGTc9aGKB/uzk7zLcQx+8pe+WmbI6I1OcgO+13LoHcvyHh5BQfrvl 1rX0m6v1jOEZXvCO6VjNJUwyGqUOCY9eps7q0+oBMLFzg9h1Mwcv1f/GJ9YkGBjk w4WbvvWo7ZOtnwTwSipNvnvvJtf9oLYayQKi5gp1bynYRwVSvucdoXD9X2E9jXpF sR1mIS6s8kagVqrkquQkiM3JZ9ZQ0/KFsRxYRmY5GO1Fcmgo8NbjSTUQ94VFxlr3 O8gXftBG5skXaUlEB0DYPwY9U0bSwD5y5FfIiSOR41SzPAKVwLbHMzGF/3o9Eg8w 4XX06dxuRvY815ex/P4OY5NPnfYIiw== =LTTz -----END PGP SIGNATURE----- --nextPart4100605.4X5qbz7Pr1--