[tarantool-patches] Re: [PATCH] Pass max_total_connections parameter to httpclient

[Kirill Yukhin <kyukhin@tarantool.org>]

Hello,

On 17 Apr 17:51, Ilya Konyukhov wrote:
> There are some usecases when current http client may leak sockets (due
> TIME_WAIT timeout). The reason is that the `max_connection` option of
> tarantool's http client (actually is a binding for CURLMOPT_MAXCONNECTS
> option in libcurl) may not be enough for some situations.
> 
> This option sets up a size of connection cache, which curl maintains
> during its lifetime. When this cache is full (i.e. all connections are
> waiting for a response from a server), newly created request would
> create a new connection to handle it. In that case right after old
> connection got response, curl has to close that connection to keep up an
> appropriate number of connections in cache. That results in a new socket
> in TIME_WAIT state (for 60 seconds by default).
> 
> This is actually easy to calculate. Imagine we have http client with
> default 8 maximum connections in cache. Also lets say we have 16384
> sockets available in the system. We need to make requests to external
> system which responses in 100ms in average. So, if we start making
> request every 11ms (100/9), every finished request will be followed by
> curl closing that socket (switches to TIME_WAIT). Result is about 90
> wasted sockets per second and additional overhead for socket creation
> for each new request.
> 
> Also if we do more than 274 requests per second, we will likely to be
> out of available sockets in 1 minute.
> 
> The solution is to add another binding for max_total_connections option,
> which will let curl hold new requests, before old ones have finished.
> It also adds more control over sockets resources.

I've checked your patch into 1.10, 2.1 and master.

--
Regards, Kirill Yukhin