From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <tarantool-patches-bounce@freelists.org>
Received: from localhost (localhost [127.0.0.1])
	by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTP id 809142E3B3
	for <tarantool-patches@freelists.org>; Tue, 14 May 2019 10:24:53 -0400 (EDT)
Received: from turing.freelists.org ([127.0.0.1])
	by localhost (turing.freelists.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id DZhVVnRhzFS0 for <tarantool-patches@freelists.org>;
	Tue, 14 May 2019 10:24:53 -0400 (EDT)
Received: from smtp37.i.mail.ru (smtp37.i.mail.ru [94.100.177.97])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by turing.freelists.org (Avenir Technologies Mail Multiplex) with ESMTPS id 3E75A2E39B
	for <tarantool-patches@freelists.org>; Tue, 14 May 2019 10:24:53 -0400 (EDT)
Date: Tue, 14 May 2019 17:24:46 +0300
From: Kirill Yukhin <kyukhin@tarantool.org>
Subject: [tarantool-patches] Re: [PATCH] Pass max_total_connections parameter
 to httpclient
Message-ID: <20190514142446.gbe4uulwshdwt2p3@tarantool.org>
References: <20190417145139.59554-1-runsfor@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20190417145139.59554-1-runsfor@gmail.com>
Sender: tarantool-patches-bounce@freelists.org
Errors-to: tarantool-patches-bounce@freelists.org
Reply-To: tarantool-patches@freelists.org
List-Help: <mailto:ecartis@freelists.org?Subject=help>
List-Unsubscribe: <mailto:tarantool-patches-request@freelists.org?Subject=unsubscribe>
List-software: Ecartis version 1.0.0
List-Id: tarantool-patches <tarantool-patches.freelists.org>
List-Subscribe: <mailto:tarantool-patches-request@freelists.org?Subject=subscribe>
List-Owner: <mailto:>
List-post: <mailto:tarantool-patches@freelists.org>
List-Archive: <http://www.freelists.org/archives/tarantool-patches>
To: tarantool-patches@freelists.org
Cc: totktonada.ru@gmail.com, Ilya Konyukhov <runsfor@gmail.com>

Hello,

On 17 Apr 17:51, Ilya Konyukhov wrote:
> There are some usecases when current http client may leak sockets (due
> TIME_WAIT timeout). The reason is that the `max_connection` option of
> tarantool's http client (actually is a binding for CURLMOPT_MAXCONNECTS
> option in libcurl) may not be enough for some situations.
> 
> This option sets up a size of connection cache, which curl maintains
> during its lifetime. When this cache is full (i.e. all connections are
> waiting for a response from a server), newly created request would
> create a new connection to handle it. In that case right after old
> connection got response, curl has to close that connection to keep up an
> appropriate number of connections in cache. That results in a new socket
> in TIME_WAIT state (for 60 seconds by default).
> 
> This is actually easy to calculate. Imagine we have http client with
> default 8 maximum connections in cache. Also lets say we have 16384
> sockets available in the system. We need to make requests to external
> system which responses in 100ms in average. So, if we start making
> request every 11ms (100/9), every finished request will be followed by
> curl closing that socket (switches to TIME_WAIT). Result is about 90
> wasted sockets per second and additional overhead for socket creation
> for each new request.
> 
> Also if we do more than 274 requests per second, we will likely to be
> out of available sockets in 1 minute.
> 
> The solution is to add another binding for max_total_connections option,
> which will let curl hold new requests, before old ones have finished.
> It also adds more control over sockets resources.

I've checked your patch into 1.10, 2.1 and master.

--
Regards, Kirill Yukhin