Running JUNIT Test Suites in a specifc order using ANT

Question

In the following example,

   1. <target name="tests.unit">  
   2.   <junit>  
      3.     <batchtest>  
      4.       <fileset dir="tsrc">  
      5.         <include name="**/Test*.java"/>  
      6.         <exclude name="**/tests/*.java"/>  
      7.       </fileset>  
      8.     </batchtest>  
      9.   </junit>  
     10. </target>  
     11. <target name="tests.integration">  
     12.   <junit>  
     13.     <batchtest>  
     14.       <fileset dir="tsrc">  
     15.         <include name="**/tests/Test*.java"/>  
     16.       </fileset>  
     17.     </batchtest>  
     18.   </junit>  
     19. </target>  

If i have several TestSuites in in the **/tests/ folder. How will it know which test suite to run first if i run the tests.integration target? If i have TestSuite1.java, TestSuite2.java and TestSuite3.java i would like the test suites to run in the order as specified in teh filename.

Answer 1

Unless there are new features in JUnit, this is a difficult thing to do.

TestNG can manage it with dependent groups.

Here's a question for you: Why does the order matter? Unit tests should not have dependencies. If you're doing that, perhaps these are really integration tests.

FitNesse might be a better way to go.

Answer 2

Yes i am trying to create a test suite for a functional test not unit tests. Im trying to use junit to build the functional tests package. I am using selenium which is based on Junit.

Lets say i have a website where you cant do anything without logging on. In this case i have a test case that tests the logging on functionality and then i would have another test case that would test something else. The order they will be executed will matter because i cant test anything before logging on which means the order should be

1. TestLogin
2. TestCreateProduct
3. TestReadProduct

in the above test cases, i cant read any product before it is created and that i have logged and i cant create a product before i have logged on. I have seen a lot of comments about using the setUp() and tearDown() methods but surely that would mean a lot of duplication.

If for example i have to make TestReadProduct test case independent, i would have to put the TestLogin and TestCreateproduct functionality in the setUp() method for the TestCreateProduct test case. Surely this is a maintenance nightmare. Imagine having to maintain 5000 of testcases. I would have to make a lot of changes in a lot of places if the TestLogin functionality changes.

I am thinking of using the "depends" option in ANT.

Something like this

<target=TestReadProduct depends=TestLogin, TestCreateProduct>

isnt there a better way of doing this?

Answer 3

You could create a base test class, put the login in its setUp() and inherit all your test cases from this one (and of course, call super.setUp() everywhere).

Note that this would only be a simple login, not a proper unit test. You should unit test your login functionality with all possible crazy user input and whatnot in a separate test class, but for the rest of the test cases you only need a plain simple login with a default username or something.

For those test cases where on top of login you need a product as well, you create a second base test class which extends the first and adds product creation to its setUp().

No code duplication - if login changes, apart from the login test cases themselves you need to change a single method in your test code.

It will probably be slower to execute 5000 unit tests this way - but much, much safer. If you start to depend on the order of execution of your unit tests, you're stepping on a slippery slope. It is very difficult to notice that you have inadvertently introduced an extra dependency between two unit tests if their order is fixed by your configuration. E.g. you set up a specific property of a product or change a global configuration setting in one unit test, then you test something else on your product in the next test case - and it happens to work only because the previous unit test set things up that specific way. This leads to nasty suprises sooner or later.