How to know about OutOfMemory or StackOverflow errors ahead of time

Question

In Java, is there a way to know that a StackOverflow error or OutOfMemory exception may happen soon?

The OutOfMemory exception might be an easier one to catch, if one is capable of getting memory usage statistics programmatically, and if one knows ahead of time how much memory needs to be used before the OutOfMemory exception is thrown. But are those values knowable?

For the StackOverflow error, is there a way to get recursion depth, and how does one know what value for recursion depth would cause the error to occur?

By knowing ahead of time whether these errors will happen, I feel I can recover the application more gracefully, instead of watching it crash.

Answer 1

You should never see a StackOverflow exception if your application is designed and implemented correctly!

Generally, if you get a StackOverflow exception then it's a sign that there's a bug in your recursion code.

Answer 2

Most StackOverflow errors come out of bad recursion. Unfortunately, the problem of determining if a recursion will stop is generally not decidable (this is a central concept in CS). There are cases, however, you could get warnings, for example some IDEs will let you know if you're invoking a function recursively with no parameters.

Answer 3

You can anticipate out-of-memory conditions with Runtime.freeMemory() and Runtime.maxMemory(). Most times it'll be hard recovering gracefully, but I leave that to you.

Answer 4

You could discover a lot about recursion depth, by creating a Throwable object, and querying its getStackTrace() method. But this is expensive to do.

If you really have a method with a small potential of throwing a StackOverflowError or an OutOfMemoryError why not just insert try-catch blocks and catch those errors? They can be caught and handled just like checked exceptions.

Answer 5

I don't know anything about working this out at run time, or what you might be able to do to avoid it once you predict it is going to happen. Better to try and avoid it occurring in the first place.

1) You could use Findbugs which may indicate some StackOverFlow errors occurring from inadvertently calling the same method from itself.

2) You could store data likely to cause you to run out of memory with a SoftReference and have a null check on accessing it so it can be reloaded if it's been garbage collected.

If either of these things are actually issues for you then the solution probably isn't in detecting it happening but architecting your solution differently to avoid them occuring if at all possible.

Answer 6

Usually memory useage is hard to predict.

Stack Overflows from infinite regression generally show up in the write/debug cycle.

Harder to catch are memory issues on things like large held collections, caches, etc.etc. As has been pointed out you can check runtime free and max memory, but be careful, as their meanings aren't obvious - "free" here, means "free now", e.g. meaning if you ran a full gc you might get more. Unfortunately there's no way to get the "total possible free, including garbage-collectible" without running System.gc(), which is not a nice thing to do on a production application (where you're liable to have large enough data sets to cause the problem in the first place) because the entire JVM will come to a screeching halt for a few seconds (or more, in a large app). Note that even System.gc() is not guaranteed to run "now", but my experience has been that it has whenever I've played with it.

You can print gc activity from the running jvm by starting java with -verbose:gc, -XX:+PrintGCTimeStamps, and -XX:+PrintGCDetails (more detail here), and in general if the collector starts to run more frequently, it's probably a sign that you're running out of memory.

Answer 7

Anticipating Out of Memory Errors

I'm surprised I didn't see this mentioned in the other posts, but you can use ManagementFactory in Java 5/6 to get at a lot of the memory usage information.

Look at the platform mbean server page for more information on detecting low memory conditions in Java. I believe you can setup notifiers to call code when memory usage reaches a certain threshold.

Answer 8

For StackOverflowError:

To know the current depth, usually it's either:

1. using a stateful function (storing the depth in outside the function)
2. using an accumulator (passing the depth as an argument to the function)

Knowing the depth it will occur is difficult. There are several factors:

1. The stack space allocated to the JVM (you can change this with the -Xss option)
2. The amount of stack space already used
3. The amount used by the current function.

Why not try it out using something like this?

public static void main(String[] args) {
    try {
        recurs();
    } catch (Throwable t) {
        // not a good idea in production code....
    }
}
static int depth = 0;
static void recurs() {
    System.out.println(depth++);
    recurs();
}

Run it several times. Also try adding dummy variables. It can be seen that even the same code may halt at different depths and adding more variables cause it to end earlier. So yeah, pretty much it's unpredictable.

I suppose that besides rewriting the algorithm the only option would be to increase the stack space with the -Xss option.

For OutOfMemoryError, there's the -Xmx option

Answer 9

One useful thing you can do is use SoftReferences for caches. That will give you a gradual performance slide as you run out of memory. Just don't use WeakReference even in a WeakHashMap because it will annoy me when your application dies nastily on me.

Answer 10

The MemoryMXBean can emit notifications if your memory reaches a certain threshold.

Answer 11

I don't know how to prevent those error conditions but if you add an unhandled exception handler to the thread at least you can log and maybe do some sort of recovery in another thread.