views:

570

answers:

3

I am running a web application on BEA Weblogic 9.2. Until recently, we were using JDK 1.5.0_04, with JAI 1.1.2_01 and Image IO 1.1. In some circumstances (we never figured out exactly why), when we were processing large images (but not that large - a few MB), the JVM would crash without any error message or stack trace or anything. This didn't happen much in production, but enough to be a nuisance and eventually we were able to reproduce it.

We decided to switch to JRockit90 1.5.0_04 and we were no longer able to reproduce the problem in our test environment, so we thought we had it licked. Now, however, after the application server has been up for a while, we start getting the error message, "Not enough storage is available to process this command" during image operations. For example:

java.lang.Error: Error starting thread: Not enough storage is available to process this command.
at java.lang.Thread.start()V(Unknown Source)
at sun.awt.image.ImageFetcher$1.run(ImageFetcher.java:279)
at sun.awt.image.ImageFetcher.createFetchers(ImageFetcher.java:272)
at sun.awt.image.ImageFetcher.add(ImageFetcher.java:55)
at sun.awt.image.InputStreamImageSource.startProduction(InputStreamImageSource.java:149)
at sun.awt.image.InputStreamImageSource.addConsumer(InputStreamImageSource.java:106)
at sun.awt.image.InputStreamImageSource.startProduction(InputStreamImageSource.java:144)
at sun.awt.image.ImageRepresentation.startProduction(ImageRepresentation.java:647)
at sun.awt.image.ImageRepresentation.prepare(ImageRepresentation.java:684)
at sun.awt.SunToolkit.prepareImage(SunToolkit.java:734)
at java.awt.Component.prepareImage(Component.java:3073)
at java.awt.ImageMediaEntry.startLoad(MediaTracker.java:906)
at java.awt.MediaEntry.getStatus(MediaTracker.java:851)
at java.awt.ImageMediaEntry.getStatus(MediaTracker.java:902)
at java.awt.MediaTracker.statusAll(MediaTracker.java:454)
at java.awt.MediaTracker.waitForAll(MediaTracker.java:405)
at java.awt.MediaTracker.waitForAll(MediaTracker.java:375)
at SfxNET.System.Drawing.ImageLoader.loadImage(Ljava.awt.Image;)Ljava.awt.image.BufferedImage;(Unknown Source)
at SfxNET.System.Drawing.ImageLoader.loadImage(Ljava.net.URL;)Ljava.awt.image.BufferedImage;(Unknown Source)
at Resources.Tools.Commands.W$zw(Ljava.lang.ClassLoader;)V(Unknown Source)
at Resources.Tools.Commands.getContents()[[Ljava.lang.Object;(Unknown Source)
at SfxNET.sfxUtils.SfxResourceBundle.handleGetObject(Ljava.lang.String;)Ljava.lang.Object;(Unknown Source)
at java.util.ResourceBundle.getObject(ResourceBundle.java:320)
at SoftwareFX.internal.ChartFX.wxvw.yxWW(Ljava.lang.String;Z)Ljava.lang.Object;(Unknown Source)
at SoftwareFX.internal.ChartFX.wxvw.vxWW(Ljava.lang.String;)Ljava.lang.Object;(Unknown Source)
at SoftwareFX.internal.ChartFX.CommandBar.YWww(LSoftwareFX.internal.ChartFX.wxvw;IIII)V(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.xxvw.YzzW(LSoftwareFX.internal.ChartFX.Internet.Server.ChartCore;Z)LSoftwareFX.internal.ChartFX.CommandBar;(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.xxvw.XzzW(LSoftwareFX.internal.ChartFX.Internet.Server.ChartCore;)V(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.ChartCore.OnDeserialization(Ljava.lang.Object;)V(Unknown Source)
at SoftwareFX.internal.ChartFX.Internet.Server.ChartCore.Zvvz(LSoftwareFX.internal.ChartFX.Base.wzzy;)V(Unknown Source)

Has anyone seen something like this before? Any clue what might be happening?

A: 

This sounds like it could be related to running out of stack space for your thread. See this post http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4765019. The JVM has a -Xss argument to control the stack size, though if you're exceeding it, it sounds like you may be creating a whole lot of threads at once...

Jeff Storey
Is that bug likely to exist in JRockit as well? If so, will the fix delivered in tiger-b05 likely apply to an updated JRockit?
jhericks
Not sure. But if this is your issue, how many threads are you creating? Can you scale that back?
Jeff Storey
We are not really creating the threads ourselves - WebLogic is. We used to tune the thread count, but since 9.0, WebLogic "auto-tunes" the thread pool, but I'm sure there's a way to scale that down.
jhericks
It would be interesting to monitor thread counts using a tool like VisualVM and see how many are live to see if it is thread related.
Jeff Storey
I'll look into it. VisualVM only works with JDK 6. Is there a similar (free) tool that you know of that will work with JDK 5?
jhericks
JConsole should
Jeff Storey
A: 

Most likely the error message is telling the truth. Either the storage mentioned refers to hard disk space? Are you sure your disks aren't running full. Otherwise you have probably run out of memory of some sort like mentioned in one of the other answers. I'm not sure about weblogic or jrockit but Sun Oracle's jdk has a nice tool called jconsole that allows you to see what's going on with your running application. It should be able to tell you about the memory consumption etc. (The tool uses jms so might even work on the other jdk's)

Paul de Vrieze
A: 

Marcus Adams should get the credit for this, but his little advice was in the form of a comment not an answer, so I can't just check it. He pointed me to this example from another answer and that did the trick.

Looking at the registry setting for \System\CurrentControlSet\Control\Session Manager\SubSystem and found this: SharedSection=1024,3072,512. Since this was a service (headless) we changed the third number. The new value was SharedSection=1024,3072,1024. The problem has not recurred since making this change a few weeks ago.

jhericks