From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from [87.239.111.99] (localhost [127.0.0.1]) by dev.tarantool.org (Postfix) with ESMTP id 784AF6EC5D; Wed, 7 Apr 2021 10:03:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 784AF6EC5D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tarantool.org; s=dev; t=1617779038; bh=dHFDyL1caQNn+tT9ZiKFS1zAJ+Dq1XozeSL1oUQaP1A=; h=Date:To:Cc:References:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=sUoF7HKnmvX54wJN5x7NJAjyH4qL1yHQ0vTLlaV9SuiZ4O8pVY+ibTxZ+NY7ydcD8 7OdRW9jPKf6UwoNENh+dMiRulER0DQBHhno4GjuILIFwA36V/ru0nPSfqG/EvLIkKt PhOkjYmwu3TNARP74VONPFdvFHvoEWJDik6ON2bs= Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by dev.tarantool.org (Postfix) with ESMTPS id 479BA6EC5D for ; Wed, 7 Apr 2021 10:03:55 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 dev.tarantool.org 479BA6EC5D Received: by mail-lf1-f45.google.com with SMTP id b14so26756875lfv.8 for ; Wed, 07 Apr 2021 00:03:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=8gVNDZ+W4eN7XiiBciSIbMXF/B0hdj8i3Ij5wsTmBh4=; b=G7y0X1xXHJQ2xnX9ApDHKquUigJB0lXFHoGilMu6oC8V1SEHP/QXkDxZEVbeohpRph EOxzDEO4sUkwND/+9zg01ag9FV/H8TqUkGDk3e9N+AL+uYPOJ3f0XKuhQ0MAVdRWOFxX 6nYab63H06X1XoUnvN1WeOlpkctlV06Y+gDUVnWFA0Bmys3wQa/B9K/hI+IBVMd9fgPi 9cztLTMXd8OyGX/1cSzhO/vwqra5dCeTeWnTZunTr53Fx4OiEAJC/FKa08XhpR6dELWi m8z/IDAYWqpjicthRDIgY/w0iB4ShCor6bwSK4T5Bp5byEGM2FoErJxlEcKwVMP6rLkT jprg== X-Gm-Message-State: AOAM531fXypShzhZZI6BCPc/Md0+H3JfRNKiAGgBP5b231tAsj7h8gvi E5bcQI65Ufz+l88wkQx83L9qgOoi/v9v2g== X-Google-Smtp-Source: ABdhPJzkFFjtzM46j2wa1j/X0A8KXk1wdcuQ58U+RNfhJEHyyD0MwgxYKW4TqCO0dLQZ+i7PZIXdnA== X-Received: by 2002:ac2:46d5:: with SMTP id p21mr1665935lfo.295.1617779034072; Wed, 07 Apr 2021 00:03:54 -0700 (PDT) Received: from grain.localdomain ([5.18.199.94]) by smtp.gmail.com with ESMTPSA id h6sm2394541lfd.77.2021.04.07.00.03.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 00:03:52 -0700 (PDT) Received: by grain.localdomain (Postfix, from userid 1000) id 1053F56015C; Wed, 7 Apr 2021 10:03:52 +0300 (MSK) Date: Wed, 7 Apr 2021 10:03:52 +0300 To: Vladislav Shpilevoy Cc: tml Message-ID: References: <20210402123420.885834-1-gorcunov@gmail.com> <20210402123420.885834-5-gorcunov@gmail.com> <14b1c3fa-da30-e7f1-6d17-32e575d71272@tarantool.org> <7a7e6003-4aac-ac7f-228d-b6f723227ea9@tarantool.org> <5bd028c8-21fe-8395-816b-1d5464aaa1ca@tarantool.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5bd028c8-21fe-8395-816b-1d5464aaa1ca@tarantool.org> User-Agent: Mutt/2.0.5 (2021-01-21) Subject: Re: [Tarantool-patches] [PATCH v20 4/7] box/module_cache: introduce modules subsystem X-BeenThere: tarantool-patches@dev.tarantool.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Tarantool development patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Cyrill Gorcunov via Tarantool-patches Reply-To: Cyrill Gorcunov Errors-To: tarantool-patches-bounces@dev.tarantool.org Sender: "Tarantool-patches" On Wed, Apr 07, 2021 at 01:43:26AM +0200, Vladislav Shpilevoy wrote: > > The module cache job is to cache, not to own. Owners are the schema > modules and box.lib modules. The cache **does not own**, therefore it > can't just delete whatever it wants. > > > And the bug is rather in caller side > > which should had install some hooks to detect exits and unref objects. > > > > But as you pointed below Lua is not properly terminated and the > > subsystem does only thing it knows about -- unref objects it has > > allocated (we setup a first ref upon allocation). It is still somehow > > ugly because of potential extra refs on Lua side and I now think > > maybe we should free allocated memory in a force way. > > As I said, under no circumstances you can free the objects which you do > not own. With this assumption OS should not release userspace memory on process termination if userspace application didn't call free(), right? OS allocated memory, cached it in pages but doesn't own it, userspace should call free first? Vlad, the key difference here is that our engine does not shutdown and then tarantool continue to work without the cache. It is exit path where we should release all memory. This as well applies to all other caches, slabs and etc. It doesn't matter that currently we simply have no way to jump into Lua internals and force clear all references sits there. I hardly convince you here but I see your point and partly agree. > > But that's > > true that even though we won't have a clean exit. I tend to agree > > that simply free and zap the hash table is best what we could do > > for now. Will update. > > I am fine with freeing the hash table itself and setting it to NULL, if > you want to free something so hard. Then at least in future it would > crash right away on any attempt to load/unload something after the cache > was destroyed. Not at some random time due to memory corruptions if you > would free the modules which you don't own and then they would be > accessed. Might happen easily if we ever would allow to load the modules > from C API, or would terminate Lua properly. Sure. I think this is the best option, will do.