From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id C4793469719 for ; Sat, 15 Feb 2020 19:34:42 +0300 (MSK) Received: by mail-lj1-f194.google.com with SMTP id w1so14092159ljh.5 for ; Sat, 15 Feb 2020 08:34:42 -0800 (PST) Date: Sat, 15 Feb 2020 19:34:40 +0300 From: Konstantin Osipov Message-ID: <20200215163440.GA19240@atlas> References: <701f259d-b9fd-c688-9602-7281e0dc2f0d@tarantool.org> <4571f818-2211-5713-fc4a-b6539ba68253@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4571f818-2211-5713-fc4a-b6539ba68253@tarantool.org> Subject: Re: [Tarantool-patches] [PATCH 0/4] Safe truncation and deletion List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vladislav Shpilevoy Cc: tarantool-patches@dev.tarantool.org * Vladislav Shpilevoy [20/02/15 19:24]: > Here is what Kostja said, but somewhy without CCing the > mailing list, and my answers to it inlined: > > > This is the first approximation of what I have proposed at best. > > First of all, the mechanism should not be truncation specific. It should be usable by all subsystems that require emergency memory. > > > > Second, there is no reason to patch small. All the patch needs to do is something along these lines: > > > > > > 1) in memtx_init, reserve the emergency slab. > > > > 2) in memtx_tuple_alloc, refuse with error if there is no emergency slab > > > > 3) in memtx_tuple_free, try to reserve emergency slab (replenish it) if it is empty > > All of this adds code and conditions to a hot path. The way with quota > enabling/disabling works only in a rare case, when delete() fails due > to OOM, or when truncate is called, which is not often. First of all, the way with quota simply doesn't work. You expect the quota to shrink back, but it never does. So you simply run out of all existing memory. If it worked, the fix would to reduce the amount of available quota by 1 slab at start, and change quota before truncate, and then return the quota back in its place. My main point: under no circumstances tarantool should go beyond the amount of memory set by quota. If we need to reserve memory/quota for emergency, it's fine, let's do it at start, but going beyond is not acceptable. Second, yes, it does add a branch to the hot path. Same as in memtx_index_extent_reserve(). This will have no impact on performance profile - feel free to check. I think, however, given the 10% performance regression in tuple_format, your time optimizing performance will be better spent elsewhere. > > 4) at start of truncate, or wherever we need emergency memory, release emergency slab. Simply return it to arena. > > Once you returned it, all the other operations will be able to take > and fill it. Such as insertions. In the first truncate or delete > didn't free anything, or freed not enough to fit a new truncate > tuple here, there is no more a reserved slab for a next delete/truncate. > So your proposal does not seem to change anything. Ehm, I did not fully explain the proposal. It also assumes there is "emergency mode" flag set at start of truncate and cleared at end, and if there is no reserve slab and there is an emergence flag set, nothing can allocate memory. -- Konstantin Osipov, Moscow, Russia