My programs run out of memory like half of the time I run them. Under Linux I can set a hard limit to the available memory using ulimit -v mem-in-kbytes. Actually, I use ulimit -S -v mem-in-kbytes, so I get a proper memory allocation problem in the program and I can abort.
But... ulimit is not working in OSX 10.6. I've tried with -s and -m options, and they are not working.
In 2008 there was some discussion about the same issue in MacRumors, but nobody proposed a good alternative. The should be a way a program can learn it's spending too much memory, or setting a limit through the OS.