Re: [Tarantool-patches] [PATCH v3] wal: introduce limits on simultaneous writes

[Konstantin Osipov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

* Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [21/02/26 10:15]:
> > I'd also question the place where you decided to put this gate.
> > The source of the issue is async requests, not WAL, which worked
> > fine in absence of async requests. So it's async requests that
> > should be gated, not WAL.
> 
> In the commit message it is clearly stated why it is in the
> journal's code, not just in the applier:
> 
> 	The feature is ready for `box.commit{is_async=true}`. Once it's
> 	implemented, it should check whether the queue is full and let the user
> 	decide what to do next. Either wait or roll the tx back.
> 
> Async transactions will be exposed to 'userspace' to be able to reduce
> latency for network requests ending with a transaction. They won't have
> to wait for WAL write to end.

You did not understand my comment. I tried to say that a major
part of this code is generic and should reside in lib/core as a
counting semaphore abstraction. Async transaction simply use this
counting semaphore to throttle themselves. Then neither WAL nor
any other resource used by async transactions will be overloaded.

Otherwise, the system would be allowed to create async
transactions, and while WAL will not overflow, some other resource
(memory, transaction identifiers, whatever) may still overflow. 

> > Otherwise your overflow will just spill out someplace else.
> 
> On the contrary. Your proposal to do it in the applier would lead to
> queue overflow in some other place - in userspace. When the queue is
> for the entire WAL, it won't overflow.

I did  not say it should be in the applier. 

-- 
Konstantin Osipov, Moscow, Russia
https://scylladb.com