Failing Android JUnit tests, not breaking my Ant script like I expect?

Question

Failing JUnit tests, not breaking my Ant script like I expect?

My continuous integration server runs an Ant script, which calls something like: /tests/ant run-tests

My JUnit tests run, but with errors: run-tests: [echo] run-tests-helper. [echo] Running tests ... [exec] [exec] com.zedray.stuff.FooBarTest:.... [exec] com.zedray.stuff.FooBarTest:.....INSTRUMENTATION_RESULT: shortMsg=Some error in your code. [exec] INSTRUMENTATION_RESULT: longMsg=java.security.InvalidParameterException: Some error in your code [exec] INSTRUMENTATION_CODE: 0

The errors are OK, but my build script keeps going (eventually publishing my broken app to my testers - bad!). What I would expect is for the instrimentaiton to throw a build error, so my continuous integration server (TeamCity in this case) realises that something has gone wrong and reports a broken build. The "failonerror" is already set in the relevant macrodef, so I'm not sure what else I can do?

/tests/build.xml

Running tests ...

Any ideas/suggestions on how to fix this?

Regards Mark

Answer 1

The ant JUnit task defaults to running all the tests. There are two solutions to this.

Easiest solution is to set the haltonerror property to true and the build will fail at the first test failure.

Slightly more involved (and my preference) is to set the failureProperty so that all the tests run. This lets you know how many tests fail instead of only the first test that fails. This requires more ant work because you need to add a line after your junit tests like this:

<fail message="tests failed" if="failureProperty"/>