[RFC v3] fiber: Increase default stack size
gorcunov at gmail.com
Tue Feb 26 14:16:32 MSK 2019
On Tue, Feb 26, 2019 at 01:26:56PM +0300, Vladimir Davydov wrote:
> Talked to Kostja and Georgy. We agreed on the following points:
> - Regarding slab poisoning. We don't want to implement ad hoc allocator
> for fiber stacks, neither do we really want to patch the small lib
> for now. Let's use madvise(MADV_DONTNEED) for all fiber stacks
> unconditionally on fiber creation. This should be okay from
> performance point of view, because once a fiber is created, it's
> never destroyed - it stays on the dead list until recycled.
> - 1 MB for max stack size seems to be a bit of an overkill for now.
> The default value should be set to 256 KB, but we do need a
> configuration option for it. Let's add it to the fiber Lua module.
> May be done in a separate patch, but should be submitted together in
> the same patch set.
Wait, first fiber for main cord is created before lua init, isn't it?
I already though about using lua config for it but fiber init'ed at
very early stage.
> - 16 byte unique identifier for detecting stack overflow doesn't seem
> to be enough. Imagine a PATH_MAX buffer allocated on stack that uses
> only a hundred bytes for path formatting. It can easily jump over the
> watermark. We should probably use random poisoning: say, 4 unique
> identifiers 8 bytes each scattered a few hundred bytes apart. Some
> math/reasoning behind this would be nice to see in the comments.
If we want to scatter we should simply put marks at page bounds.
Dirtifying somewhere inside middle of a page is useless.
> - Since madvise() is somewhat expensive to be called on each fiber
> recycle, we need to make the watermark dynamic. That is, keep track
> of the number of fibers that have exceeded the watermark and when
> there are too many of those, increase the watermark value. We could
> probably use a histogram for this. This would allow us to decrease
> the default stack allocation size from 64 KB down to 16 KB, which
> would be really nice. This should be done in a separate patch, but
> again in the scope of this issue.
> - We definitely need this patch (or patches) to be covered with unit
> tests. Please add corresponding cases to test/unit/fiber.cc.
Yes, need to address this too. Thanks a huge for comments!
More information about the Tarantool-patches