JPA/HSQLDB is still eating all my memory

Question

Hi guys,

I'm working on a program that parses data from a file and stores it into a HSQLDB database. As the parser encounters data it creates entities which are mapped to the database using JPA/Hibernate. My problem is that while the parsing is being performed the application uses more and more memory. I have successfully used cached tables so that once the parsing is complete the memory is all released, but during the parsing it uses way more than what I am comfortable with.

I have tried to fix this by calling flush and clear methods on my EntityManager, but this had no effect. I've also tried to make sure that Entity with references to all other entities is kept in memory.

The largest objects in memory seems to be hsqldb.Sessions. Could it be that HSQlDb caches loads of data for each transaction? It does seem excessive to need 1GB of RAM to only end up with a DB that is 120MB on disk does it not?

Please advise on what I could try next.

:)

Answer 1

Do a heap dump and use Eclipse MAT to analyse where the memory is used. With JPA the results are often surprising and without looking at the actual memory usage you're often stabbing in the dark.

Answer 2

After two days of faffing around with HSQLDB I followed the advice of two friends and changed database to H2. The memory footprint during a transaction is roughly a third now and it is also 20% faster.

Really surprised me