Testing with Spring - Avoiding false positives?

Question

In the spring documentation regarding testing, it states:

Avoid false positives when testing ORM code

When you test code involving an ORM framework such as JPA or Hibernate, flush the underlying session within test methods which update the state of the session. Failing to flush the ORM framework's underlying session can produce false positives: your test may pass, but the same code throws an exception in a live, production environment. In the following Hibernate-based example test case, one method demonstrates a false positive and the other method correctly exposes the results of flushing the session.

Can someone explain why i need to call flush?

Answer 1

Well, you actually skipped the interesting part, the example :) Here it is:

// ...

@Autowired
private SessionFactory sessionFactory;

@Test // no expected exception!
public void falsePositive() {
    updateEntityInHibernateSession();
    // False positive: an exception will be thrown once the session is
    // finally flushed (i.e., in production code)
}

@Test(expected = GenericJDBCException.class)
public void updateWithSessionFlush() {
    updateEntityInHibernateSession();
    // Manual flush is required to avoid false positive in test
    sessionFactory.getCurrentSession().flush();
}

// ...

What this example tries to illustrate is that unless you actually flush the session (A.K.A. the first level cache) to sync in-memory changes with the database, you're not really testing the database integration and might not test the real expected behavior or miss a problem.

For example, the database could return an error because of, say a constraint violation, and if you don't hit the database, you won't exhibit this right behavior, as in the falsePositive() test method above. This test method should fail, or expect an exception but will just pass. On the other hand, the other test method with the flush does test the real behavior. Hence the need to flush.

Answer 2

Spring documentation uses the wrong concept. It has been clear

but the same code throws an exception in a live, production environment

Here goes wikipedia

Type II error, also known as an "error of the second kind", a β error, or a "false negative": the error of failing to reject a null hypothesis when it is in fact not true. An example of this would be if a test shows that a woman is not pregnant, when in reality, she is.

If you see the sample provided by Spring, The production environment throws an exception (A GenericJDBCException), but it has not been detected. In order to see, you must call the underlying commit when using some ORM provider.

XUnit Test patterns definition

A situantion in which a test passes even though the system under test is not working properly.

So the right concept is falseNegative

@Test // no expected exception!
public void falseNegative() {