From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 367476EC56; Tue, 16 Mar 2021 09:45:43 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 367476EC56 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1615877143; bh=YijOuf5lWZhLxuRwA4OJZ6HZ+LgGKo2YYxmmor6BB8Q=; h=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=BYgZJ20L2y1P0QMJwu5EaZT1K1ClTof0B6ZUdPPY1e5fkPfvbjBSuMn8eEbB6X+ab FRXkio7CIiy6bVjewOAtJ0fIUTF7rKdiXO+QV0R1OLx7tomOkVCo7fN0VvbNd7mIIq P8IRMD47FLo7nzqBdSj6Q0agvj9y/twN8jKeZoyc= Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 70B6D6EC56 for ; Tue, 16 Mar 2021 09:45:42 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 70B6D6EC56 Received: by mail-lf1-f44.google.com with SMTP id u4so60747293lfs.0 for ; Mon, 15 Mar 2021 23:45:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=6IKVPn5lwftwpfErf/AH+LRtosw19tYjRfZV8fWG2Bw=; b=c0uALrHv/HxGChOuGq+bLdvqDjJt9qNFrgMttBIaWHiL3iR/qWZzbDBdm4Vqhvmg83 4sKtZu5zHlWduULY35OXtFarTxupauXdov7UIivZE9w8eiqsHiWvMC3xFFuIy+druk1U G56kVZmgyFFQQG+AI0zYgnthotgRb93uaI13/46p1XLKOhx9MsIzhwRHQ5pahHnOZuVv f2V01GTQrItDYOwhvZ3Bd3FB1tvl0ZnajYYuKYWUL0v8YTDUj2Fxnj+Y3MS0yT+9hDoY /O2DTzBZ2af80nQxy8dRdrbGsc18Ab6ALkU+AZ8JnxhGTWkn2uRYlVVnViuwXZWoyXHB 6PWA== X-Gm-Message-State: AOAM533h3qUnjIfdbqGlacqrqOrsWNiJM4cNcU6VSHo8MzWqd5/UsymF dM2ZUxs3wYx1N526IMgBhVNKJoSxYA== X-Google-Smtp-Source: ABdhPJwv+zBP/0NuIOWY8yO6XwsfbSEcIPISlNtTG0kMvyCgYfJX3f7dmULEpk5ayE7i+hPaG3uZhA== X-Received: by 2002:a05:6512:3045:: with SMTP id b5mr10087259lfb.32.1615877141751; Mon, 15 Mar 2021 23:45:41 -0700 (PDT) Received: from sterling.local ([46.188.68.12]) by smtp.gmail.com with ESMTPSA id z21sm3013908ljh.104.2021.03.15.23.45.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Mar 2021 23:45:41 -0700 (PDT) Received: by sterling.local (Postfix, from userid 1000) id D8652E60068; Tue, 16 Mar 2021 09:45:40 +0300 (MSK) Date: Tue, 16 Mar 2021 09:45:40 +0300 To: Vladislav Shpilevoy Message-ID: <20210316064540.GA39305@starling> Mail-Followup-To: Konstantin Osipov , Vladislav Shpilevoy , Serge Petrenko , gorcunov@gmail.com, tarantool-patches@dev.tarantool.org References: <20210224193549.70017-1-sergepetrenko@tarantool.org> <26fde1bf-a972-fe03-fffe-818839718394@tarantool.org> <20210310081804.GA87351@starling> <54666e16-f577-c9f5-e917-f834b7e4b8d5@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54666e16-f577-c9f5-e917-f834b7e4b8d5@tarantool.org> Subject: Re: [Tarantool-patches] [PATCH v3] wal: introduce limits on simultaneous writes X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Konstantin Osipov via Tarantool-patches Reply-To: Konstantin Osipov Cc: tarantool-patches@dev.tarantool.org Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" * Vladislav Shpilevoy [21/03/16 09:16]: BTW, one option is to change semantics of "async" to be "best effort async". That is, if the queue is full, we don't drop the transaction, we turn it into a waiting one. Thoughts? > Hi! Thanks for the patch! > > I must admit, it looks kind of ugly :D. The class we have now only remotely > looks like a semaphore. > > Number of reasons, some of which you already listed in the header file: > > - It is advisory. Can be bypassed easily if you forget to check wouldblock. > But not a big issue really. An optional thing like 'try_take' is needed > for box.commit({is_async = true}) anyway, not to block the fiber; > > - You can take more amount of the resource than there is. Bearable as well, > but still; > > - sem_release() does not wakeup anybody. Quite counter-intuitive; > > - The wouldblock check not only checks the resource being available, but also > if there are any waiters. It wouldn't matter for a real semaphore, because > it has nothing to do with ordering the waiters in FIFO. It is a detail of > the journal which slipped into the general class. > But maybe that is the only way to make it fair? Otherwise some fibers > could be blocked forever due to starvation. > > The last thing I am not sure is even an issue. Might be a feature. > > The others probably can be fixed if we would rework journal_queue API. For > instance, not have journal_queue_wait() separated from journal_queue_on_append(). > Then sem_take() could become blocking and obligatory. > > You simply inline everything into journal_write() and journal_write_try_async(), > and you will see that you can always call take() and block inside of it. > > But I don't know if it is worth doing TBH. It is used in a single place so far. > This is hard to define fiber_sem API which would be suitable for future usages. > I would vote for not doing it now and see if we would need the semaphore in the > future. > > Although the idea about removing journal_queue_wait() might be worth trying. > It is used either right before journal_queue_on_append(), or in > journal_queue_flush() which is also right before journal_queue_on_append(). > Up to you. Anyway we need to return to this code for box.commit({is_async}) > feature, which means the hard polishing might be not so useful. -- Konstantin Osipov, Moscow, Russia https://scylladb.com