Re: [Tarantool-patches] [PATCH v20 4/7] box/module_cache: introduce modules subsystem

[Cyrill Gorcunov via Tarantool-patches <tarantool-patches@dev.tarantool.org>]

On Wed, Apr 07, 2021 at 01:43:26AM +0200, Vladislav Shpilevoy wrote:
> 
> The module cache job is to cache, not to own. Owners are the schema
> modules and box.lib modules. The cache **does not own**, therefore it
> can't just delete whatever it wants.
> 
> > And the bug is rather in caller side
> > which should had install some hooks to detect exits and unref objects.
> > 
> > But as you pointed below Lua is not properly terminated and the
> > subsystem does only thing it knows about -- unref objects it has
> > allocated (we setup a first ref upon allocation). It is still somehow
> > ugly because of potential extra refs on Lua side and I now think
> > maybe we should free allocated memory in a force way.
> 
> As I said, under no circumstances you can free the objects which you do
> not own.

With this assumption OS should not release userspace memory on process
termination if userspace application didn't call free(), right? OS allocated
memory, cached it in pages but doesn't own it, userspace should call free
first? Vlad, the key difference here is that our engine does not shutdown
and then tarantool continue to work without the cache. It is exit path where
we should release all memory. This as well applies to all other caches,
slabs and etc. It doesn't matter that currently we simply have no way
to jump into Lua internals and force clear all references sits there.

I hardly convince you here but I see your point and partly agree.

> > But that's
> > true that even though we won't have a clean exit. I tend to agree
> > that simply free and zap the hash table is best what we could do
> > for now. Will update.
> 
> I am fine with freeing the hash table itself and setting it to NULL, if
> you want to free something so hard. Then at least in future it would
> crash right away on any attempt to load/unload something after the cache
> was destroyed. Not at some random time due to memory corruptions if you
> would free the modules which you don't own and then they would be
> accessed. Might happen easily if we ever would allow to load the modules
> from C API, or would terminate Lua properly.

Sure. I think this is the best option, will do.