views:

204

answers:

2

So I've noticed that the default stack size for threads on linux is 8MB (if I'm wrong, PLEASE correct me), and, incidentally, 1MB on Windows. This is quite bad for my application, as on a 4-core processor that means 64 MB is space is used JUST for threads! The worst part is, I'm never using more than 100kb of stack per thread (I abuse the heap a LOT ;)).

My solution right now is to limit the stack size of threads. However, I have no idea how to do this portably. Just for context, I'm using Boost.Thread for my threading needs. I'm okay with a little bit of #ifdef hell, but I'd like to know how to do it easily first.

Basically, I want something like this (where windows_* is linked on windows builds, and posix_* is linked under linux builds)

// windows_stack_limiter.c
int limit_stack_size()
{
    // Windows impl.
    return 0;
}

// posix_stack_limiter.c
int limit_stack_size()
{
    // Linux impl.
    return 0;
}

// stack_limiter.cpp
int limit_stack_size();
static volatile int placeholder = limit_stack_size();

How do I flesh out those functions? Or, alternatively, am I just doing this entirely wrong? Remember I have no control over the actual thread creation (no new params to CreateThread on Windows), as I'm using Boost.Thread.

+4  A: 

You do not need to do this. The machine's physical memory is employed only where it is needed by a demand page fault system. Even if the thread stacks are significantly larger than the amount you are using the extra size is in virtual address space and does not tie up physical RAM.

Had physical RAM been tied up at that rate, a typical machine would run out of memory with only a few dozen processes running. You can see from a ps -Al that quite a few more than that execute concurrently.

Amardeep
So it's really nothing to worry about? 8MB is a scary number.
Clark Gaebel
Every process gets a couple of gigabytes of virtual address space all to itself so unless you know you're allocating close to that I wouldn't think you'll have a problem. I've run hundreds of threads in a process with no issue.
Amardeep
I usually hit this limit on Linux when running a few hundred threads because it actually exhausts the two gigabytes of process address space. At which time we've added ulimit -s 1024 to the startup scripts.
Cubbi
A: 

First, you don't need to change this unless you are getting SEGVs from hitting this limit. (see man setrlimit for detailed info)

Second, you change this in all of the linux distributions I'm aware of by editing /etc/security/limits.conf (to change the default) or by running ulimit -s <stack size in kilobytes> to change the value until you exit the shell.

Slartibartfast
Whups, answered the wrong question. Sorry.
Slartibartfast