From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 442E62586E for ; Fri, 16 Aug 2019 09:53:34 -0400 (EDT) Received: from turing.freelists.org ([127.0.0.1]) by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VClQgqmN4qvq for ; Fri, 16 Aug 2019 09:53:34 -0400 (EDT) Received: from smtp47.i.mail.ru (smtp47.i.mail.ru [94.100.177.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 0464624940 for ; Fri, 16 Aug 2019 09:53:33 -0400 (EDT) Date: Fri, 16 Aug 2019 16:53:32 +0300 From: Konstantin Osipov Subject: [tarantool-patches] Re: [PATCH 1/7] Refactoring: wal writer fiber and queue Message-ID: <20190816135331.GB18643@atlas> References: <921a4cbb0f93c43954e4b18a0d61ac6dd4a563ea.1565676868.git.georgy@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <921a4cbb0f93c43954e4b18a0d61ac6dd4a563ea.1565676868.git.georgy@tarantool.org> Sender: tarantool-patches-bounce@freelists.org Errors-to: tarantool-patches-bounce@freelists.org Reply-To: tarantool-patches@freelists.org List-Help: List-Unsubscribe: List-software: Ecartis version 1.0.0 List-Id: tarantool-patches List-Subscribe: List-Owner: List-post: List-Archive: To: tarantool-patches@freelists.org Cc: Georgy Kirichenko * Georgy Kirichenko [19/08/13 10:30]: > As wal processes all writes in a cbus loop fiber it isn't possible to > yield while write. The patch introduces a wal write queue and a wal write > fiber which fetch a batch from queue and writes it out. Also checkpoint > requests are going now throw the queue to synchronize a tx checkpoint though > status with wal. I don't understand this patch. First, WAL already runs in a fiber, and you can yield. cord_costart creates a new fiber, please check out the implementation. this is done by this commit: commit f4625e64eb99c17910d3f0bcd323e5d82b6d5b31 Author: Vladimir Davydov Date: Wed Jul 4 20:20:55 2018 +0300 vinyl: use cbus for communication between scheduler and worker threads We need cbus for forwarding deferred DELETE statements generated in a worker thread during primary index compaction to the tx thread where they can be inserted into secondary indexes. Since pthread mutex/cond and cbus are incompatible by their nature, let's rework communication channel between the tx and worker threads using cbus. Needed for #2129 Check out the commit, it changes cord_start() to cord_costart(). Second, checkpoint message already flushed the queue. So I don't understand why you need a switch over message type. Let's discuss online. -- Konstantin Osipov, Moscow, Russia