From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 0BE2B6EC5D; Mon, 5 Apr 2021 17:03:27 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 0BE2B6EC5D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1617631407; bh=YPVMlCF75uHXytw1nx10y8cruEdFrfi6oLVjWBcNlNU=; h=To:Cc:References:Date:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=yif8TFvFXw/2EJVl+G1U9vjoZzPEzhW6+aayX1F7VmEeX0PyqRCdVtkvqXlR4qKa9 BE6U1onpmrlXEvsO6wq/fLnaWq2LQkgsC1b+87qLt2DSs7DAb95Yb1h26+PjBT8Djb +RGmmyZrDSTCUPEaDqz/rFWDT0GCTQtx9w5o3ODE= Received: from smtp33.i.mail.ru (smtp33.i.mail.ru [94.100.177.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id E0D386EC5D for ; Mon, 5 Apr 2021 17:03:24 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org E0D386EC5D Received: by smtp33.i.mail.ru with esmtpa (envelope-from ) id 1lTPp6-0006TU-0c; Mon, 05 Apr 2021 17:03:24 +0300 To: v.shpilevoy@tarantool.org, alexander.turenko@tarantool.org Cc: tarantool-patches@dev.tarantool.org References: <44f4b260e0ac435fe78282c1f6f4607159914935.1617375300.git.sergepetrenko@tarantool.org> Message-ID: <6216debc-5807-ede1-df21-823d60a1e70b@tarantool.org> Date: Mon, 5 Apr 2021 17:03:23 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <44f4b260e0ac435fe78282c1f6f4607159914935.1617375300.git.sergepetrenko@tarantool.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-7564579A: B8F34718100C35BD X-77F55803: 4F1203BC0FB41BD9ED7173E37F4E3294CA3588DDE0233B0D17711AF1EA2D7DB9182A05F538085040250FAE991DA07DE404E1CA433249171CE9A27311DECA48A9ED0B480EC2B979A6 X-7FA49CB5: FF5795518A3D127A4AD6D5ED66289B5278DA827A17800CE706EA9E10470DC775EA1F7E6F0F101C67BD4B6F7A4D31EC0BCC500DACC3FED6E28638F802B75D45FF8AA50765F7900637AAF0FBE7E77B7ED78638F802B75D45FF914D58D5BE9E6BC131B5C99E7648C95C16EE06F5A270FE6A6187269283E484B637B05D5FC82B62A0A471835C12D1D9774AD6D5ED66289B5278DA827A17800CE74A95F4E53E8DCE969FA2833FD35BB23D2EF20D2F80756B5F868A13BD56FB6657A471835C12D1D977725E5C173C3A84C3E97D2AE7161E217F117882F4460429728AD0CFFFB425014E868A13BD56FB6657D81D268191BDAD3DC09775C1D3CA48CFB88863A07377B48BBA3038C0950A5D36C8A9BA7A39EFB766EC990983EF5C0329BA3038C0950A5D36D5E8D9A59859A8B66EA1BA7CA28B4A743AA81AA40904B5D9DBF02ECDB25306B2201CA6A4E26CD07C3BBE47FD9DD3FB595F5C1EE8F4F765FC72CEEB2601E22B093A03B725D353964B0B7D0EA88DDEDAC722CA9DD8327EE4930A3850AC1BE2E735B7122A4C44A4E42FC4224003CC83647689D4C264860C145E X-C1DE0DAB: 0D63561A33F958A5A08F3F56C28E1F97E188FA17E7B37E8895361F0EB0C6F225D59269BC5F550898D99A6476B3ADF6B47008B74DF8BB9EF7333BD3B22AA88B938A852937E12ACA7502E6951B79FF9A3F410CA545F18667F91A7EA1CDA0B5A7A0 X-C8649E89: 4E36BF7865823D7055A7F0CF078B5EC49A30900B95165D348D68DCC07DD06FF5D61865582F26EB874004AADCCE2FBF12B7C22C0F7A04E85D2BD90844B9A40B9A1D7E09C32AA3244CCCFBE6ABE9DC7E6487C18870F7577968A8CE788DE6831205FACE5A9C96DEB163 X-D57D3AED: 3ZO7eAau8CL7WIMRKs4sN3D3tLDjz0dLbV79QFUyzQ2Ujvy7cMT6pYYqY16iZVKkSc3dCLJ7zSJH7+u4VD18S7Vl4ZUrpaVfd2+vE6kuoey4m4VkSEu530nj6fImhcD4MUrOEAnl0W826KZ9Q+tr5ycPtXkTV4k65bRjmOUUP8cvGozZ33TWg5HZplvhhXbhDGzqmQDTd6OAevLeAnq3Ra9uf7zvY2zzsIhlcp/Y7m53TZgf2aB4JOg4gkr2biojM00ve/f+0omCKXDEP8G3bQ== X-Mailru-Sender: 583F1D7ACE8F49BDD2846D59FC20E9F81F116986441EF4D53BEF22477BD0E4BEE91E5C0C426EA6BC424AE0EB1F3D1D21E2978F233C3FAE6EE63DB1732555E4A8EE80603BA4A5B0BC112434F685709FCF0DA7A0AF5A3A8387 X-Mras: Ok Subject: Re: [Tarantool-patches] [PATCH 4/4] feedback_daemon: generate report right before sending X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Serge Petrenko via Tarantool-patches Reply-To: Serge Petrenko Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" 02.04.2021 17:58, Serge Petrenko пишет: > Feedback daemon used to generate report before waiting (for an hour by > default) until it's time to send it. Better actualize the reports and > generate them right when it's time to send them. > > Follow-up #5750 > --- > src/box/lua/feedback_daemon.lua | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/src/box/lua/feedback_daemon.lua b/src/box/lua/feedback_daemon.lua > index 2ce768642..0aab189b6 100644 > --- a/src/box/lua/feedback_daemon.lua > +++ b/src/box/lua/feedback_daemon.lua > @@ -327,12 +327,13 @@ local function feedback_loop(self) > fiber.name(PREFIX, { truncate = true }) > > while true do > - local feedback = self:generate_feedback() > local msg = self.control:get(self.interval) > -- if msg == "send" then we simply send feedback > if msg == "stop" then > break > - elseif feedback ~= nil then > + end > + local feedback = self:generate_feedback() > + if feedback ~= nil then > pcall(http.post, self.host, json.encode(feedback), {timeout=1}) > end > end Pushed a new commit on top: ===================================== Bump `feedback_version` to 7 and introduce a new field: `feedback.events`. It holds a counter for every event we may choose to register later on. Currently the possible events are "create_space", "drop_space", "create_index", "drop_index". All the registered events and corresponding counters are sent in a report in `feedback.events` field. Also, the first registered event triggers the report sending right away. So, we may follow such events like "first space/index created/dropped" Closes #5750 ---  src/box/lua/feedback_daemon.lua       | 26 ++++++++++++++++++++++++--  src/box/lua/schema.lua                | 16 ++++++++++++++++  test/box-tap/feedback_daemon.test.lua | 23 ++++++++++++++++++++++-  3 files changed, 62 insertions(+), 3 deletions(-) diff --git a/src/box/lua/feedback_daemon.lua b/src/box/lua/feedback_daemon.lua index 0aab189b6..ca93b7574 100644 --- a/src/box/lua/feedback_daemon.lua +++ b/src/box/lua/feedback_daemon.lua @@ -312,13 +312,18 @@ local function fill_in_stats(feedback)      feedback.stats = stats  end -local function fill_in_feedback(feedback) +local function fill_in_events(self, feedback) +    feedback.events = self.cached_events +end + +local function fill_in_feedback(self, feedback)      fill_in_base_info(feedback)      fill_in_platform_info(feedback)      fill_in_repo_url(feedback)      fill_in_features(feedback)      fill_in_options(feedback)      fill_in_stats(feedback) +    fill_in_events(self, feedback)      return feedback  end @@ -358,12 +363,26 @@ local function guard_loop(self)      self.shutdown:put("stopped")  end +local function save_event(self, event) +    if type(event) ~= 'string' then +        error("Usage: box.internal.feedback_daemon.save_event(string)") +    end +    local cnt = self.cached_events[event] or 0 +    self.cached_events[event] = (self.cached_events[event] or 0) + 1 +    if self.cached_events[event] == 1 then +        -- The first occurred event of this type triggers report dispatch +        -- immediately. +        self.send() +    end +end +  -- these functions are used for test purposes only  local function start(self)      self:stop()      if self.enabled then          self.control = fiber.channel()          self.shutdown = fiber.channel() +        self.cached_events = {}          self.guard = fiber.create(guard_loop, self)      end      log.verbose("%s started", PREFIX) @@ -402,7 +421,7 @@ setmetatable(daemon, {          end,          -- this function is used in saving feedback in file          generate_feedback = function() -            return fill_in_feedback({ feedback_version = 6 }) +            return fill_in_feedback(daemon, { feedback_version = 7 })          end,          start = function()              start(daemon) @@ -413,6 +432,9 @@ setmetatable(daemon, {          reload = function()              reload(daemon)          end, +        save_event = function(event) +            save_event(daemon, event) +        end,          send = function()              if daemon.control ~= nil then                  daemon.control:put("send") diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index 96503a50e..cf1b070af 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -462,6 +462,10 @@ box.schema.space.create = function(name, options)      })      _space:insert{id, uid, name, options.engine, options.field_count,          space_options, format} + +    if internal.feedback_daemon ~= nil then +        internal.feedback_daemon.save_event("create_space") +    end      return box.space[id], "created"  end @@ -531,6 +535,9 @@ box.schema.space.drop = function(space_id, space_name, opts)              box.error(box.error.NO_SUCH_SPACE, space_name)          end      end +    if internal.feedback_daemon ~= nil then +        internal.feedback_daemon.save_event("drop_space") +    end  end  box.schema.space.rename = function(space_id, space_name) @@ -1237,6 +1244,11 @@ box.schema.index.create = function(space_id, name, options)          local _func_index = box.space[box.schema.FUNC_INDEX_ID]          _func_index:insert{space_id, iid, index_opts.func}      end + +    if internal.feedback_daemon ~= nil then +        internal.feedback_daemon.save_event("create_index") +    end +      return space.index[name]  end @@ -1257,6 +1269,10 @@ box.schema.index.drop = function(space_id, index_id)          _func_index:delete({v.space_id, v.index_id})      end      _index:delete{space_id, index_id} + +    if internal.feedback_daemon ~= nil then +        internal.feedback_daemon.save_event("drop_index") +    end  end  box.schema.index.rename = function(space_id, index_id, name) diff --git a/test/box-tap/feedback_daemon.test.lua b/test/box-tap/feedback_daemon.test.lua index 3d6bf1d9b..a2e041649 100755 --- a/test/box-tap/feedback_daemon.test.lua +++ b/test/box-tap/feedback_daemon.test.lua @@ -70,7 +70,7 @@ if not ok then      os.exit(0)  end -test:plan(28) +test:plan(30)  local function check(message)      while feedback_count < 1 do @@ -293,5 +293,26 @@ check_stats(actual.stats)  actual = daemon.generate_feedback()  test:is(box.info.uptime, actual.uptime, "Server uptime is reported and is correct.") +daemon.reload() +actual = daemon.generate_feedback() + +local events_expected = {} +test:is_deeply(actual.events, events_expected, "Events are empty initially.") + +box.schema.space.create('test') +box.space.test:create_index('pk') +box.space.test.index.pk:drop() +box.space.test:drop() + +actual = daemon.generate_feedback() +events_expected = { +    create_space = 1, +    create_index = 1, +    drop_space = 1, +    drop_index = 1, +} + +test:is_deeply(actual.events, events_expected, "Events are counted correctly") +  test:check()  os.exit(0) -- 2.24.3 (Apple Git-128) -- Serge Petrenko