From: Vladimir Davydov <vdavydov.dev@gmail.com>
To: Cyrill Gorcunov <gorcunov@gmail.com>
Cc: "Георгий Кириченко" <georgy@tarantool.org>,
tarantool-patches@freelists.org
Subject: Re: [RFC v3] fiber: Increase default stack size
Date: Tue, 26 Feb 2019 13:26:56 +0300 [thread overview]
Message-ID: <20190226102656.gwwy35jyaqdkci3l@esperanza> (raw)
In-Reply-To: <20190226091254.GL7198@uranus>
On Tue, Feb 26, 2019 at 12:12:54PM +0300, Cyrill Gorcunov wrote:
> > > I'm not yet familiar with slab engine, does it allocates
> > > pages on lazy fashion or we need to pass 'dontneed' on
> > > first fiber creation too?
> >
> > Oops, you're right, good catch! The allocator may poison slab if NDEBUG
> > is unset. So we can either
> >
> > - Madvise slab on fiber creation, at least in NDEBUG mode. Simple, but
> > depends on the allocator internals.
> > - Patch the 'small' library to make the allocator do madvise for us.
> > IMO it would look better, but would clutter the allocator API.
> > - Don't use 'small' allocator at all for default slab allocations, and
> > simply mmap stack and link them in a free list (is it OK to mmap a
> > few MB chunk per each fiber?).
> >
> > I'm inclined to choose the last option. I'll discuss the options with
> > others today and follow-up.
>
> Will continue at the evening. Also note that stack is special and
> we better should not pass arbitrary sizes to allocate even for
> custom stacks, they all _must_ be PAGE_SIZE aligned (or call it
> PAGE_SIZE orders if you prefer :) Anything else make madvise
> idea useless since kernel requires madvise args to be page
> aligned which is understandable due to hardware.
Talked to Kostja and Georgy. We agreed on the following points:
- Regarding slab poisoning. We don't want to implement ad hoc allocator
for fiber stacks, neither do we really want to patch the small lib
for now. Let's use madvise(MADV_DONTNEED) for all fiber stacks
unconditionally on fiber creation. This should be okay from
performance point of view, because once a fiber is created, it's
never destroyed - it stays on the dead list until recycled.
- 1 MB for max stack size seems to be a bit of an overkill for now.
The default value should be set to 256 KB, but we do need a
configuration option for it. Let's add it to the fiber Lua module.
May be done in a separate patch, but should be submitted together in
the same patch set.
- 16 byte unique identifier for detecting stack overflow doesn't seem
to be enough. Imagine a PATH_MAX buffer allocated on stack that uses
only a hundred bytes for path formatting. It can easily jump over the
watermark. We should probably use random poisoning: say, 4 unique
identifiers 8 bytes each scattered a few hundred bytes apart. Some
math/reasoning behind this would be nice to see in the comments.
- Since madvise() is somewhat expensive to be called on each fiber
recycle, we need to make the watermark dynamic. That is, keep track
of the number of fibers that have exceeded the watermark and when
there are too many of those, increase the watermark value. We could
probably use a histogram for this. This would allow us to decrease
the default stack allocation size from 64 KB down to 16 KB, which
would be really nice. This should be done in a separate patch, but
again in the scope of this issue.
- We definitely need this patch (or patches) to be covered with unit
tests. Please add corresponding cases to test/unit/fiber.cc.
next prev parent reply other threads:[~2019-02-26 10:26 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-22 20:16 [tarantool-patches] [RFC v2] " Cyrill Gorcunov
2019-02-25 14:55 ` Vladimir Davydov
2019-02-25 15:25 ` Cyrill Gorcunov
2019-02-25 21:39 ` [RFC v3] " Cyrill Gorcunov
2019-02-26 8:58 ` Vladimir Davydov
2019-02-26 9:12 ` Cyrill Gorcunov
2019-02-26 10:26 ` Vladimir Davydov [this message]
2019-02-26 10:36 ` Vladimir Davydov
2019-02-26 11:17 ` Cyrill Gorcunov
2019-02-26 12:25 ` Vladimir Davydov
2019-02-26 11:16 ` Cyrill Gorcunov
2019-02-26 12:34 ` Vladimir Davydov
2019-02-26 12:54 ` Cyrill Gorcunov
2019-02-26 13:06 ` Vladimir Davydov
2019-02-26 13:26 ` [tarantool-patches] " Konstantin Osipov
2019-02-26 14:02 ` Cyrill Gorcunov
2019-02-26 10:32 ` Vladimir Davydov
2019-02-26 11:18 ` Cyrill Gorcunov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190226102656.gwwy35jyaqdkci3l@esperanza \
--to=vdavydov.dev@gmail.com \
--cc=georgy@tarantool.org \
--cc=gorcunov@gmail.com \
--cc=tarantool-patches@freelists.org \
--subject='Re: [RFC v3] fiber: Increase default stack size' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox