OutOfMemoryError: Trying to optimise Maven multi-module unit tests using HSQLDB, DBUnit, unitils, Hibernate, Spring

Question

I have a big multi-module Maven project with thousands of tests. Each test loads DAOs, services, etc using the SpringApplicationContext annotation. My unitils configuration looks like this:

database.driverClassName=org.hsqldb.jdbcDriver
database.url=jdbc:hsqldb:file:mytestdb
database.schemaNames=PUBLIC
database.userName=sa
database.password=
database.dialect=hsqldb
unitils.modules=database,dbunit,hibernate,inject,spring
# custom version of HsqldbDbSupport which calls "SET REFERENTIAL_INTEGRITY FALSE"
org.unitils.core.dbsupport.DbSupport.implClassName.hsqldb=my.company.unitils.HsqldbDbSupport
org.dbunit.dataset.datatype.IDataTypeFactory.implClassName.hsqldb=org.dbunit.ext.hsqldb.HsqldbDataTypeFactory
org.unitils.dbmaintainer.structure.ConstraintsDisabler.implClassName=org.unitils.dbmaintainer.structure.impl.DefaultConstraintsDisabler
# custom version of CleanInsertLoadStrategy which calls "DatabaseUnitils.disableConstraints();"
DbUnitModule.DataSet.loadStrategy.default=my.company.unitils.DataSetLoadStrategy
HibernateModule.configuration.implClassName=org.hibernate.cfg.AnnotationConfiguration
dbMaintainer.disableConstraints.enabled=true
dbMaintainer.cleanDb.enabled=true
dbMaintainer.generateDataSetStructure.enabled=false
dataSetStructureGenerator.xsd.dirName=target/xsd
dbMaintainer.autoCreateExecutedScriptsTable=true
updateDataBaseSchema.enabled=false
dbMaintainer.fromScratchEnabled=true

My Hibernate configuration looks like this:

hibernate.hbm2ddl.auto=update
hibernate.show_sql=false
hibernate.format_sql=false
hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.connection.autocommit=false
hibernate.cache.provider_class=org.hibernate.cache.NoCacheProvider
hibernate.cache.use_query_cache=false
hibernate.cache.use_second_level_cache=false
hibernate.cache.use_structured_entries=false
hibernate.generate_statistics=false

When I run the tests, the 2GB allocated for heap eventually fills up. Permgen reaches about 400MB. Looking at the heap dump, char, java.lang.String and java.lang.String[] takes up about 40% of the total instances.

When I spot-check a few of these instances, the GC root traces back to:

• the Spring context manager (in the form of context XML extracts)
• the SessionFactory (in the form of queries)

I would like to optimise these 2 things, and I think they are respectively caused by:

• Each test has a @SpringApplicationContext annotation
• Each test also uses the unitils @DataSet annotation to load up data for that test.

Note, I use jdbc:hsqldb:file to offload some data off of the heap (i.e. instead of using jdbc:hsqldb:mem)

Versions:
hsqldb 1.8.0.7
Spring 3.0.4.RELEASE
unitils 3.1

PS can someone with a 1500+ rating please tag this question with "unitils"?

Answer 1

If the memory is taken by the data stored in the database, you can make the database use disk based tables to reduce memory use. This can be specified on the database URL:

database.url=jdbc:hsqldb:file:mytestdb;hsqldb.default_table_type=cached