Java Servlet Excel export refreshes the request every 300 seconds - even when the first one isn't done yet

Question

I'm working on a program that often does Excel exports from hypermassive database tables. One test case I'm doing right now does about 300k rows by 40 columns into one xlsx worksheet. It completes in about 6 minutes or so, taking about 1.1GB RAM, but I've run into a major hangup - after exactly 300 seconds have passed, I see this pattern start in the logs:

2010-10-04 13:15:13.201 ID0000 GRP=| Executing Excel Export    
2010-10-04 13:20:13.156 ID0000 GRP=| Executing Excel Export    
2010-10-04 13:21:25.041 ID0000 GRP=| Excel Output Completed
2010-10-04 13:25:13.114 ID0000 GRP=| Executing Excel Export 
2010-10-04 13:26:34.315 ID0000 GRP=| Excel Output Completed
2010-10-04 13:30:13.074 ID0000 GRP=| Executing Excel Export
2010-10-04 13:31:28.854 ID0000 GRP=| Excel Output Completed

Essentially, it executes the same request after almost exactly 300 seconds, thus doubling my RAM usage (to a very uncomfortable 2GB) and delaying the response while driving the CPU use up as it ties up another core. In addition, because the request has refreshed, the browser doesn't catch the results of the first request, which as you can see completed shortly after making the second request. So the user sees no output at all while the server takes more and more resources.

I have replicated this behavior in IE8 and in Firefox, and tried using response.setHeader("Refresh", "999999"); in the servlet class.

We are using Aspose.Cells to do the export, but it seems to be unrelated to anything that library is doing, as I can confirm that the entire request is being repeated from beginning to end, not just the call to Aspose.

Anybody know if there's a property I can set, either on our weblogic server or within the browser, that stops this behavior?

Edit: I can confirm that this happens even after the browser window has been closed - I still keep seeing "Executing Excel Export" messages every 5 minutes. They essentially continue until I restart the weblogic server.

Answer 1

This sounds like your request is taking more time than actual timeout. I know that EJB RMI request have 300sec timeout in weblogic be default. If you are using EJB (or some other weird stuff that uses RMI), I suggest you change timeout-config. If you dont use EJBs, I think the best is to look for some weblogic request config with this timeout value.

Answer 2

Turns out we had Apache on the backend trying to do some sort of load balancing when servlet requests take too long, but since there were no other servers it would just make the same request again, exacerbating the problem. I set the WLIOTimeout in httpd.conf and httpd-ssl.conf to a higher number and we don't have the problem anymore. Thanks.