Best testing framework?

Question

Is JUnit the best unit testing framwork? Which framework is best for new java projects?

Answer 1

I would not say there is any which could be described as best. But the two choices I know about are JUnit (version 3 and 4) and TestNG. JUnit is integrated into the Eclipse, so it was easier to use for me from the beginning, as I neededmore featres I started to use version 4. However, my colleague in work prefers TestNG and from one podcast I listened to, I guess TestNG is more feature rich.

Answer 2

It is not the testing framework that counts, but having good test cases.

Answer 3

Best is relative to need.

JUnit is standard, and the best supported. There are many extensions that rely on it, so in many projects you are likely to end up using it anyway. JUnit focuses exclusively on unit testing, though. Some find that an advantage.

TestNG is well supported as well, but isn't quite as ubiquitous as JUnit. But the advantage of TestNG is that it is much more flexible so it can be leveraged well for tests that have inter-dependencies (integration and functional tests), something that JUnit avoids by design. I have also found that if you want to organize a one-to-one relationship between your test and the class under test, TestNG gives you much better organizational abilities. The JUnit answer would be to make static nested classes for that kind of case (say where some tests were parameterized and some needed a different kind of runner).

Personally I think TestNG's XML configuration for running tests is kind of lame, but some people still love XML configuration (and it certainly can be very advantageous in advanced scenarios) so it is a point to consider.

I think for a new project you should be thinking about one of those two first, and only branch out from there if there is a specific feature you find that they are lacking.

Answer 4

You might also consider FitNesse and Selenium for your testing toolbox. SOAP UI is great for testing web services. JMeter or Grinder for poor man's load testing.

As always, it's the right tool for the job.

Answer 5

Every one has his/her preferences. I agree with Yishai that you should use the best tools for your job but even then I have a bias towards TestNG as compared to JUnit. I have found that TestNG is often the best tool for my testing needs. There are my reasons why.

1. TestNG has various configuration possibilities as per your needs If you want one time setUp/teardown before and after a test suite @BeforeSuite/AfterSuite annotations can be used. I hate that JUnit equates one time initialization as static. If I have a state that I only need to initialize once (say a database connection), I have it static. Static state in your test code can be a pain. This is not a problem with TestNG.
2. The notion of skipped tests. If you have a bunch of core tests that need to be successful before the dependent tests are executed, then you are setting yourself for trouble if one of the test fails: In JUnit all dependent tests would be considered as a failures. This is not the case with TestNG. TestNG skips the dependent tests. This is true for configuration methods as well.
3. Parameterized Tests: Any one who has dealt with legacy code will know that a class under test have many methods accepting various types of arguments. TestNG data provider configuration allows you to set up different parameterized arguments for different test methods. JUnit supports parameterized testing by passing arguments through constructor. If you have multiple sets of test data (say 3), JUnit creates 3 objects of the test class, each time passing a different sets of parameters. TestNG creates one test object and iterates through the data set in a loop.

There are other features that I love but this is what I love the most.