From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 0448F2D6C7 for ; Fri, 30 Nov 2018 03:58:14 -0500 (EST) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R_BOTdalhq5k for ; Fri, 30 Nov 2018 03:58:13 -0500 (EST) Received: from smtp49.i.mail.ru (smtp49.i.mail.ru [94.100.177.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 93EF92D6B6 for ; Fri, 30 Nov 2018 03:58:12 -0500 (EST) Received: from [185.6.245.178] (port=46400 helo=atlas.local) by smtp49.i.mail.ru with esmtpa (envelope-from ) id 1gSeck-0001mY-H8 for tarantool-patches@freelists.org; Fri, 30 Nov 2018 11:58:10 +0300 Date: Fri, 30 Nov 2018 11:58:10 +0300 From: Konstantin Osipov Subject: [tarantool-patches] Re: [PATCH 7/9] box: rewrite checkpoint daemon in C Message-ID: <20181130085810.GG5760@chai> References: <3a95ae5e66ebfd72125eb53afb914efb5ab9cc0a.1543419109.git.vdavydov.dev@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3a95ae5e66ebfd72125eb53afb914efb5ab9cc0a.1543419109.git.vdavydov.dev@gmail.com> Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-help: List-unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-subscribe: List-owner: List-post: List-archive: To: tarantool-patches@freelists.org * Vladimir Davydov [18/11/28 19:16]: > Long time ago, when the checkpoint daemon was added to Tarantool, it was > responsible not only for making periodic checkpoints, but also for > maintaining the configured number of checkpoints and removing old snap > and xlog times, so it was much easier to implement it in Lua than in C. files > However, over time, all its responsibilities have been reimplemented in > C and moved to the server code so that now it just calls box.snapshot() > periodically. Let's rewrite this simple procedure in C as well - this > will allow us to easily add more complex logic there, e.g. triggering > checkpoint when WAL files exceed a configured threshold. > --- > src/box/CMakeLists.txt | 1 - > src/box/box.cc | 102 ++++++++++++++++++++++++ > src/box/box.h | 1 + > src/box/lua/cfg.cc | 12 +++ > src/box/lua/checkpoint_daemon.lua | 136 -------------------------------- > src/box/lua/init.c | 2 - > src/box/lua/load_cfg.lua | 2 +- > test/xlog/checkpoint_daemon.result | 145 ----------------------------------- Could we please move this fiber to gc.c at least? Let's not pollute box? > + if (box_checkpoint_is_in_progress) { > + /* > + * The next checkpoint will be scheduled > + * by the concurrent box_checkpoint(). > + */ This is rather fragile. Can we make the interaction between box-checkpoint and checkpoint daemon less obvious? Imagine in future we can perform checkpoints according to a cron. I think it would be better if the logic of next checkpoint time calculation is consolidated in the checkpoint daemon alone. If the daemon sees that a checkpoint is in progress it can skip the current checkpoint, but not delay the next checkpoint till infinity. > + next_checkpoint_time = now + TIMEOUT_INFINITY; > + continue; > + } > + box_checkpoint(); > + } > + checkpoint_daemon = NULL; > + return 0; > +} > + > --- a/test/xlog/checkpoint_daemon.test.lua > +++ b/test/xlog/checkpoint_daemon.test.lua Please keep the test. I don't understand what's wrong with it, neither it is obvious from changeset comments. -- Konstantin Osipov, Moscow, Russia, +7 903 626 22 32 http://tarantool.io - www.twitter.com/kostja_osipov