Why are my DBUnit tests consuming so much memory?

Question

I've got a hibernate-based application which uses DBUnit for unit testing. We have an XML test database, which gets loaded up with dummy data in the setUp() of each test and deleted during the tearDown(). The problem is that I can no longer run the entire suite in an IDE (in this case, Intellij), because after about 300 tests, the heap memory gets all used up. The tests go from taking ~0.3 seconds to 30+ seconds to execute, until the JVM eventually gives up and dies.

When I run the test suite via ant's junit task, then it's no problem, nor is running the test suite for an individual class. However, I like being able to run the whole suite locally before I check in large refactoring changes to the codebase rather than breaking the build on the CI server.

I am running the test suite with -Xmx512m as my only argument to the JVM, which is the same amount I pass to ant when running the task on the CI server. My hibernate-test.cfg.xml looks like this:

<hibernate-configuration>
  <session-factory>
    <!-- Database connection settings -->
    <property name="connection.driver_class">org.hsqldb.jdbcDriver</property>
    <property name="connection.url">jdbc:hsqldb:mem:mydatabase</property>
    <property name="connection.username">sa</property>
    <property name="connection.password"/>

    <!-- Other configuration properties -->
    <property name="connection.pool_size">1</property>
    <property name="jdbc.batch_size">20</property>
    <property name="connection.autocommit">true</property>
    <property name="dialect">org.hibernate.dialect.HSQLDialect</property>
    <property name="current_session_context_class">thread</property>
    <property name="cache.provider_class">org.hibernate.cache.HashtableCacheProvider</property>
    <property name="bytecode.use_reflection_optimizer">false</property>
    <property name="show_sql">true</property>
    <property name="hibernate.hbm2ddl.auto">create-drop</property>

    <!-- Mappings (omitted for brevity) -->
    <mapping resource="hbm/blah.hbm.xml"/>
  </session-factory>
</hibernate-configuration>

We have written a class for which all of the test classes extend from, which looks something like this:

package com.mycompany.test;
// imports omitted for brevity

public abstract class DBTestCase extends TestCase {

  private final String XML_DATA_SET = "test/resources/mytestdata.xml";
  private Session _session;
  private Configuration _config;

  public DBTestCase(String name) {
    super(name);
  }

  @Override
  protected void setUp() throws Exception {
    super.setUp();
    _config = new Configuration().configure();
    SessionFactory sf = _config.buildSessionFactory();
    // This is a singleton which is used the DAO's to acquire a session.
    // The session must be manually set from the test's setup so that any
    // calls to the singleton return this session factory, otherwise NPE
    // will result, since the session factory is normally built during
    // webapp initialization.
    HibernateUtil.setSessionFactory(sf);
    _session = sf.openSession();
    _session.beginTransaction();

    IDataSet dataSet = new FlatXmlDataSet(new File(XML_DATA_SET));
    DatabaseOperation.CLEAN_INSERT.execute(getConnection(), dataSet);
  }

  protected void tearDown() throws Exception {
    super.tearDown();
    _session.close();
  }

  protected IDatabaseConnection getConnection() throws Exception {
    ConnectionProvider connProvider = ConnectionProviderFactory
      .newConnectionProvider(_config.getProperties());
    Connection jdbcConnection = connProvider.getConnection();
    DatabaseConnection dbConnection = new DatabaseConnection(jdbcConnection);
    DatabaseConfig dbConfig = dbConnection.getConfig();
    dbConfig.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY, new HsqldbDataTypeFactory());
    return dbConnection;
  }
}

It is clear that some memory leak is going on here, but I'm not sure where. How might I go about diagnosing this?

Answer 1

You are using memory database here:

<property name="connection.driver_class">org.hsqldb.jdbcDriver</property>
<property name="connection.url">jdbc:hsqldb:mem:mydatabase</property>

That means everything in the database is in the memory. Either use on disk database with cached table, or make sure you drop everything after each test.

Answer 2

J-16 SDiZ's answer got me working in the right direction, but I thought I would provide a bit more detailed information as to how I was able to solve this. The root of the problem was indeed that the database kept being stored in memory, but the solution was to inherit from DBUnit's DBTestCase class, not try to roll my own by inheriting from the JUnit TestCase. My test case base class now looks something like this:

public class MyTestCase extends DBTestCase {
  private static Configuration _config = null;

  public MyTestCase(String name) {
    super(name);
    if(_config == null) {
      _config = new Configuration().configure();
      SessionFactory sf = _config.buildSessionFactory();
      HibernateUtil.setSessionFactory(sf);
    }

    System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_DRIVER_CLASS, "org.hsqldb.jdbcDriver");
    System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_CONNECTION_URL, "jdbc:hsqldb:mem:mydbname");
    System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_USERNAME, "sa");
    System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_PASSWORD, "");
  }

  @Override
  protected IDataSet getDataSet() throws Exception {
    return new FlatXmlDataSet(new FileReader(MY_XML_DATA_FILE_NAME), false, true, false);
  }

  @Override
  protected void setUpDatabaseConfig(DatabaseConfig config) {
    config.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY, new HsqldbDataTypeFactory());
  }

This class works quite well, and my test suite runs have gone down from several minutes to a mere 30 seconds.