From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id DA2A66EC6F; Fri, 26 Feb 2021 10:18:32 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org DA2A66EC6F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1614323912; bh=GdiK37UztTV77ZiHBrtrkQkcXzP7rav7qjh0rxps4OU=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=p3VQOegKL9BZnNpc+VJsPSyYItAqcoY/+/6Aqje0DBfUvEyq+D4MWVC23gmB2N0TZ 2Vcgscr+Y1E7b9IkaYenvOh8JISuryImOPb6WaZwbhrUq1J+C/n0k44PohLk8G5BYb x9bl6bH42Rsan83e8XEUWq7r8GZ0F07jfM6ZMO4k= Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C2DD26EC6F for ; Fri, 26 Feb 2021 10:18:31 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org C2DD26EC6F Received: by mail-lf1-f42.google.com with SMTP id b1so1644321lfb.7 for ; Thu, 25 Feb 2021 23:18:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=yJhnyixhf30fjQpyp7PEgkF7b8X/nJ8F3t44i9j5wSc=; b=AgZbCSBVkn02Uqx6JLTmPosN7Y9bgUiNzPSm+yV/Id0Lyw5RrycIZE27IIw/v7TVhE EHnYpWHDkuMUACaCYIelEMea5FlicKgp8xmEHSZJjSJJ/3qkMruKU7gkfjuj65n9D6dZ t7rUN3fyRidwvQqmbqvEzH+1XuPjlEP/6hxmQWCKruGJET+G1/m8ajFobndOpmxACGLh qCOgBoBSdUNWRek1dIJfupCQd6d4Is/Egox9k6DDoADpfxUtYlppBnt3FYvUBD5iZ+0P FrVmUA4VXQimwRZI6DW19DBpTNTHz/gI99yNNKzoKf9lojjvdsb4C2nDsP9Y95bJFjSY 1qhQ== X-Gm-Message-State: AOAM533kDw1LSNan8YUObb6Pa7ZYkEVIsQp8PWtRN/bXQAp9hF1oULUJ IL6I2O6WchvK23HxtwH/RWLDwlCUdQ== X-Google-Smtp-Source: ABdhPJwIki53HfC8zb0UgJhNxCKQrw3Hcr66YU4WNFxG9Oh9LTw2mm81WQ6itw19jPjB8rbEjs7zHw== X-Received: by 2002:a19:810c:: with SMTP id c12mr1044709lfd.244.1614323911017; Thu, 25 Feb 2021 23:18:31 -0800 (PST) Received: from sterling.local ([81.25.56.224]) by smtp.gmail.com with ESMTPSA id c9sm1351813ljn.99.2021.02.25.23.18.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 23:18:30 -0800 (PST) Received: by sterling.local (Postfix, from userid 1000) id 2A61AE60067; Fri, 26 Feb 2021 10:18:29 +0300 (MSK) Date: Fri, 26 Feb 2021 10:18:29 +0300 To: Vladislav Shpilevoy Message-ID: <20210226071829.GD18388@starling> Mail-Followup-To: Konstantin Osipov , Vladislav Shpilevoy , Serge Petrenko , gorcunov@gmail.com, tarantool-patches@dev.tarantool.org References: <20210224193549.70017-1-sergepetrenko@tarantool.org> <20210225130514.GB18388@starling> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Tarantool-patches] [PATCH v3] wal: introduce limits on simultaneous writes X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Konstantin Osipov via Tarantool-patches Reply-To: Konstantin Osipov Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" * Vladislav Shpilevoy [21/02/26 10:15]: > > I'd also question the place where you decided to put this gate. > > The source of the issue is async requests, not WAL, which worked > > fine in absence of async requests. So it's async requests that > > should be gated, not WAL. > > In the commit message it is clearly stated why it is in the > journal's code, not just in the applier: > > The feature is ready for `box.commit{is_async=true}`. Once it's > implemented, it should check whether the queue is full and let the user > decide what to do next. Either wait or roll the tx back. > > Async transactions will be exposed to 'userspace' to be able to reduce > latency for network requests ending with a transaction. They won't have > to wait for WAL write to end. You did not understand my comment. I tried to say that a major part of this code is generic and should reside in lib/core as a counting semaphore abstraction. Async transaction simply use this counting semaphore to throttle themselves. Then neither WAL nor any other resource used by async transactions will be overloaded. Otherwise, the system would be allowed to create async transactions, and while WAL will not overflow, some other resource (memory, transaction identifiers, whatever) may still overflow. > > Otherwise your overflow will just spill out someplace else. > > On the contrary. Your proposal to do it in the applier would lead to > queue overflow in some other place - in userspace. When the queue is > for the entire WAL, it won't overflow. I did not say it should be in the applier. -- Konstantin Osipov, Moscow, Russia https://scylladb.com