From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Vladimir Davydov Subject: [PATCH 0/2] Rate limit certain warining messages Date: Tue, 11 Dec 2018 19:31:16 +0300 Message-Id: To: kostja@tarantool.org Cc: tarantool-patches@freelists.org List-ID: Certain warning messages can be printed quite often, e.g. "too long WAL write" is printed by each fiber stalled on the WAL. If there are hundreds of fibers and the IO subsystem becomes unresponsive from time to time, the log may grow to the scale of gigabytes pretty quickly, which complicates investigation of real problems. Also, say() (the function used for logging) is quite CPU consuming. E.g. in certain cases when readahead or net_msg_max is misconfigured, "stopping input" messages may easily eat all available CPU time and make the instance unresponsive. To overcome those problems, this patch set introduces say_ratelimited() helper, which is just like say() except it suppresses a messages if too many messages of the same kind have already been printed in the last few seconds. The new helper is used for all kinds of messages that may spam the log. https://github.com/tarantool/tarantool/issues/2218 https://github.com/tarantool/tarantool/commits/dv/gh-2218-rate-limit-certain-log-messages Vladimir Davydov (2): Introduce basic rate limiter Rate limit certain warning messages src/box/iproto.cc | 10 +++-- src/box/txn.c | 5 ++- src/box/vy_point_lookup.c | 7 +-- src/box/vy_quota.c | 5 ++- src/box/vy_read_iterator.c | 9 ++-- src/ratelimit.h | 104 +++++++++++++++++++++++++++++++++++++++++++++ src/say.h | 38 +++++++++++++++++ test/unit/CMakeLists.txt | 2 + test/unit/ratelimit.c | 78 ++++++++++++++++++++++++++++++++++ test/unit/ratelimit.result | 13 ++++++ 10 files changed, 256 insertions(+), 15 deletions(-) create mode 100644 src/ratelimit.h create mode 100644 test/unit/ratelimit.c create mode 100644 test/unit/ratelimit.result -- 2.11.0