Web application with Hibernate, Spring and Oracle goes up to 99% of the CPU (row lock contentions).

views:

362

answers:

Web application with Hibernate, Spring and Oracle goes up to 99% of the CPU (row lock contentions).

Hi there.

We have an web application in Linux environment that the CPU sometimes goes up to 99%.

Sometimes it takes days, and other times it takes minutes. We are using Hibernate with Spring in a tomcat webapp and an Oracle Database.

Checking the logs it appears the following:

"ConnectionManager - transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!".

And then the sessioncount starts to grow until 256 sessions (the max allowed by our Apache confs). This is the line that appears when the session count gets 256:

"ContainerBackgroundProcessor[StandardEngine[Catalina]] ManagerBase - Start expire sessions StandardManager at 1259947978384 sessioncount 256"

After that the CPU gets 99%.

Any suggestions? All will be HIGHLY appreciated.

Thanks in advance.

p.s. Sometime ago we have some similar problems with row lock contention. That's why we adopt Spring.

A few details, are you using JRun or are you using Tomcat? It would really help if you can detail out your environment.

Benjamin Wong 2009-12-03 15:30:58

You're right. I'm using Tomcat.

foxtrot 2009-12-03 15:48:06

Hmmm, this looks like a code issue. How many applications do you have running there? Could be that one of your applications is running amok retrieving data or doing stuff.Also have you tried running your CF server without Hibernate running? It could be hibernate updating its cache.Hope this helps.

Benjamin Wong 2009-12-03 16:03:18

It only has one application. We've tried only CF like you say and it worked ok for several days. :) Yeahhhh!! Interesting what you've say about hibernate cache..how can I troubleshoot that?? Thks a lot.

foxtrot 2009-12-03 16:14:21

They are many things that can go awry with Hibernate. I am not sure about how to exactly do it because I don't really touch my Hibernate stuff but from what I understand, Hibernate actually does a lot of caching of data and depending on how big is your database, how often those queries are retrieved and how your HQL queries are like, Hibernate will cache accordingly. How big is your database? It could be that the hibernate cache you have is quite big and it is updating the cache. I know this is not a direct answer but its the best I can give you. Hope there are better Hibernate SO experts here

Benjamin Wong 2009-12-04 01:09:21

I were checking with our hibernate guys and we don't use hibernate cache (by the way our DB is Oracle).In catalina.out log it shows this string several times increasing until the CPU goes 99%:12/03 13:50:53 Information [scheduler-2] - Run Client Storage PurgeWhat is throwing this to the log? Coldfusion? And if it is, is possible that is not provoqued by Coldfusion, is only Coldfusion informing that don't have memory?

foxtrot 2009-12-04 15:03:36

See the answer from Sam Farmer below, his method might be your ticket to solving the issue. Interesting that this does not crop up when you don't have Hibernate running though.

Benjamin Wong 2009-12-05 04:48:21

There isn't enough information to truely know what's causing your issue here. Some information that would be helpful would be information about the number coldfusion threads(and maximum). The database load is also something that would be meaningful, when the web server load spikes what is the database doing. What changed in conjunction with this problem coming into play?

np0x 2009-12-04 03:17:14

np0x I was checking what you said about "what is the database doing" and the logs, and it appears the following "ConnectionManager - transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!". And the sessioncount starts to grow (ContainerBackgroundProcessor[StandardEngine[Catalina]] ManagerBase - Start expire sessions StandardManager at 1259947978384 sessioncount 62) until 256 (the max allowed by our Apache confs).

foxtrot 2009-12-04 18:10:15

Then the CPU goes to 99%!

foxtrot 2009-12-04 18:16:13

After investigating this "ConnectionManager - transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!" should I configure the following: "<prop key="hibernate.connection.release_mode">after_statement</prop>" in hibernate properties config XML? Any hibernate expert out there? Thks in advance!

foxtrot 2009-12-05 12:51:09

Ah, it might be the dreaded client purge issue. If you are not using client variables turn them off.

From the CFAdmin > Server Settings > Client Variables > Select Default Storage Mechanism for Client Sessions set to None.

Sam Farmer 2009-12-04 21:38:33

We already desactivate that Sam. After checking this link: http://kb2.adobe.com/cps/000/ce14cfd1.html. The problems keep appearing.

foxtrot 2009-12-05 08:36:22

ansaurus

tags:

views:

answers:

Web application with Hibernate, Spring and Oracle goes up to 99% of the CPU (row lock contentions).

related questions