Is there a way to release a saturated connection pool?

Question

Hi,

I'm working with open esb on a glassfish server. We have a connection pool that works with an as400 Database.

Every couple of days we get this error: Error in allocating a connection. Cause: In-use connections equal max- pool-size and expired max-wait-time. Cannot allocate more connections

The best way to relief the cp is to restart the server. We've also managed to set to another cp with the same attributes.

My questions is: Is there a way to actively "tell" the cp to release all of its open connections?

Cheers, Eran

Answer 1

Before doing so, figure out why the connections are not released properly. Sounds like there is a single place where this is forgotten (do you have all close() in finally clauses?).

I can strongly recommend Michael Nygards "Release It!" for techniques to make software production ready.

EDIT#1: If I understand your description correctly your backend programs go in MSGW in QSYSOPR which results in a hanging connection until the response is given which in your case is close to never. Is it an option to use a profile with a default reply of "C" which allows the fault to propagate to you as an exception?

Otherwise you may be able to set a timeout period for either the connections or for the whole server of 24 hours? Then at least the connections will eventually be closed. This solution does not scale though, but may make development easier.

Please note, it is also possible to have a separate surveillance thread which looks for MSGW's regularily and send them an answer automatically AFTER grabbing the call back stack for post mortem analysis.

Answer 2

If you can determine an upper bound for "normal" connection usage in seconds you can use GlassFish' connection leakage detection mechanism. In GF's admin console (I use v2.1) go to Resources / JDBC / Connection Pools / [your cp] / Advanced and under Connection Settings set Leak Reclaim to true and set a time in seconds for Leak Timeout.

Answer 3

You mentioned in your comment to Andersen that you're getting AS400 messages thrown. It's possible for you to set up automatic answers to these messages to avoid leaving the exception message open. Check out the WRKRPYLE (Work Reply List Entry) command on the AS400 to automatically reply to these errors and avoid a hang.

Answer 4

I had this problem and it turned out to be the Transaction Management. Adding @TransactionManagement(TransactionManagementType.BEAN) to the class solved the problem. In my case, I didn't want any transaction code, but your mileage may vary depending on your requirements.