Tarantool development patches archive
 help / color / mirror / Atom feed
From: Vladislav Shpilevoy via Tarantool-patches <tarantool-patches@dev.tarantool.org>
To: Konstantin Osipov <kostja.osipov@gmail.com>,
	Serge Petrenko <sergepetrenko@tarantool.org>,
	gorcunov@gmail.com, tarantool-patches@dev.tarantool.org
Subject: Re: [Tarantool-patches] [PATCH v3] wal: introduce limits on simultaneous writes
Date: Fri, 26 Feb 2021 23:44:31 +0100	[thread overview]
Message-ID: <e0d4b68e-4e0b-f1d6-b1bc-9b2b66f36f0b@tarantool.org> (raw)
In-Reply-To: <20210226212034.GE84448@starling>

On 26.02.2021 22:20, Konstantin Osipov wrote:
> * Vladislav Shpilevoy <v.shpilevoy@tarantool.org> [21/02/26 23:24]:
>> Talking of the other limits - firstly we need to find if some of them
>> really overflows. Then yes, such a semaphone-thing could be applied
>> there too. But AFAIK, there are no other known similar bugs yet.
> 
> Exploring this rather theoretically, since there are no user async
> transactions yet, I can imagine such transaction takes up memory
> and then blocks on WAL semaphore. If there is no limit on the
> number of async transactions, it can be a lot of memory. On the
> other hand this can be limited by a yet another semaphore.

Yes, such transactions will only occupy memory. But the plan is return
an error from box.commit({is_async}) if the WAL queue is full already.
Because we don't want to block the async commit in anyway. Better bail
out earlier and give the user a chance to call normal box.commit() if
necessary. Or introduce some kind of 'try_async' to block if the queue
is full, but no block if not full, I don't know. We didn't think on the
API yet.

There will be a limit both on number of transactions and on their
total size. The limits are introduced right in this patch, and will
be used by async transactions in the future.

The main usage, if I understand correctly, will be for latency-sensitive
applications, when you send your transactions/calls/IPROTO commands via
network, send your data to WAL and return response to the client
immediately. You don't have to wait even until writev() completes.

Interestingly, it won't lead to loss of any guarantees, because anyway
completion of writev() does not mean anything. Still can power off before
it hits the disk, and still the node can fail loosing all non-replicated
data. So seems like a good feature.

Especially should be good for synchrononus transactions, where commit
duration can be quite big.

  reply	other threads:[~2021-02-26 22:44 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-24 19:35 Serge Petrenko via Tarantool-patches
2021-02-24 19:40 ` Serge Petrenko via Tarantool-patches
2021-02-25 13:05 ` Konstantin Osipov via Tarantool-patches
2021-02-26  0:57   ` Vladislav Shpilevoy via Tarantool-patches
2021-02-26  7:18     ` Konstantin Osipov via Tarantool-patches
2021-02-26 20:23       ` Vladislav Shpilevoy via Tarantool-patches
2021-02-26 21:20         ` Konstantin Osipov via Tarantool-patches
2021-02-26 22:44           ` Vladislav Shpilevoy via Tarantool-patches [this message]
2021-02-27 13:27             ` Konstantin Osipov via Tarantool-patches
2021-03-01 19:15   ` Serge Petrenko via Tarantool-patches
2021-03-01 21:46     ` Konstantin Osipov via Tarantool-patches
2021-02-26  0:56 ` Vladislav Shpilevoy via Tarantool-patches
2021-03-01 19:08   ` Serge Petrenko via Tarantool-patches
2021-03-01 22:05     ` Vladislav Shpilevoy via Tarantool-patches
2021-03-02 17:51       ` Serge Petrenko via Tarantool-patches
2021-03-03 20:59         ` Vladislav Shpilevoy via Tarantool-patches
2021-03-09 15:10           ` Serge Petrenko via Tarantool-patches
2021-03-09 19:49 ` Vladislav Shpilevoy via Tarantool-patches
2021-03-10  8:18   ` Konstantin Osipov via Tarantool-patches
2021-03-12 17:10     ` Serge Petrenko via Tarantool-patches
2021-03-13 19:14       ` Konstantin Osipov via Tarantool-patches
2021-03-15 23:42       ` Vladislav Shpilevoy via Tarantool-patches
2021-03-16  6:45         ` Konstantin Osipov via Tarantool-patches
2021-03-16 20:27           ` Vladislav Shpilevoy via Tarantool-patches
2021-03-16 10:19         ` Serge Petrenko via Tarantool-patches
2021-03-16 20:48           ` Vladislav Shpilevoy via Tarantool-patches
2021-03-17 12:14             ` Serge Petrenko via Tarantool-patches
2021-03-17 21:02           ` Vladislav Shpilevoy via Tarantool-patches
2021-03-19 11:32             ` Serge Petrenko via Tarantool-patches
2021-03-19 15:36 ` Kirill Yukhin via Tarantool-patches

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e0d4b68e-4e0b-f1d6-b1bc-9b2b66f36f0b@tarantool.org \
    --to=tarantool-patches@dev.tarantool.org \
    --cc=gorcunov@gmail.com \
    --cc=kostja.osipov@gmail.com \
    --cc=sergepetrenko@tarantool.org \
    --cc=v.shpilevoy@tarantool.org \
    --subject='Re: [Tarantool-patches] [PATCH v3] wal: introduce limits on simultaneous writes' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox