Getting suggestions on how to get the best code coverage in java

Question

Several tools exists that allow to calculate path coverage for a set of tests, but is there a tool (or an algorithm) that could suggest values to get the best path coverage with the smallest number of tests possible?

For example with the following classes:

public class User {

    private boolean isAdmin = false;
    private String name;
    private String password;

    public User(String name, String password, boolean isAdmin) {
        this.name = name;
        this.password = password;
        this.isAdmin = isAdmin;
    }

    public boolean isAdmin() {
        return isAdmin;
    }

    public boolean authenticate(String name, String password) {
        if (name.equals(this.name) && password.equals(this.password)) {
        return true;
        } else {
            return false;
        }
    }
}

---

public class DataRepository {

    List<String> data = new ArrayList<String>();

    public void add(String dataPiece) {
        data.add(dataPiece);
    }

    public void clearAll(User userAuthenticated) {
        if (userAuthenticated.isAdmin()) {
            data.clear();
        }
    }

    public void addAll(User userAuthenticated, List<String> collection) {
        if (userAuthenticated.isAdmin()) {
            data.addAll(collection);
        }
    }
}

I will get a better test coverage if i create a test which is using a User with isAdmin at true.

Is there a tool that could mention: if you create a test with User with isAdmin at true you will get the best test coverage.

Something like that but for more complicated cases, and that checks all the branches of code.

Answer 1

What "values" are you talking about suggesting?

Code coverage works by inspecting which lines/methods/classes etc are executed during a particular session. This is typically used when running automated unit tests, but it needn't be (e.g. you can instrument a webapp before deploying it for smoke testing, to get an idea of what code paths you didn't cover in your ad-hoc tests).

The way to increase coverage is simply to make sure you're calling more methods with arguments that execute more of the internal branches, within the context of whatever it is you're measuring (e.g. unit tests). No tool is going to tell you exactly what to write to do this - they highlight what hasn't been tested, and it's up to you to write the test that will do so.

Unless your question was "which coverage tool gives the highest measurements for coverage", in which case it's a bad criterion for assessing suitability.

EDIT - you also need to realise that the goal of tests is not to get good coverage scores. Code coverage is only useful to show what code is definitely not tested. A class can still have 100% coverage without having any useful tests (e.g. trivially, ones without any assertions - or more usually, ones whose assertions also pass for some failing values).

Thus if you write tests solely with the intention of getting coverage up, your tests will be garbage and not give you anything. Write the tests initially thinking solely about the functionality of the class, and only when you think "I'm done", run it through a coverage tool. Use the output of the tool to let you see blocks of code that haven't been exercised, which may suggest you need to write more tests passing in different inputs. And above all, when writing tests always be sure of what it is you're testing and what output should constitute a pass and fail.

Answer 2

Clover has the feature that it shows you the poorly tested, highly complex code. This is good so you don't waste time increasing code coverage by testing getters and setters.

Many tools can calculate the cyclomatic complexity of a function. If this value is to high, you just not going to be able to test all the permutations of the function.

Code coverage is a means - not an end. Code coverage does not prove code is correct, and the coverage tool is only useful as the bugs it helps you uncover.

Answer 3

Test Driven Development. If you want to get the best possible coverage, you could try test driving out all the code you write. It requires a lot of discipline and is even better when you pair program, but if you do it correctly, I would expect 100% code coverage.

Another tip is to minimise the number of unit tests you write where you test a single class atomically and thus stubbing out other classes. The most powerful of tests (e.g. system tests or acceptance tests or whatever you wish to call them) are those that start as high up in the chain of classes (right at the entry point of your application if possible) and then test without stubbing or mocking anything. At worst you would be running a complete system talking to just an in-memory database such as hsqldb containing your test data. Writing tests such as these would help reduce the number of test classes (and therefore tests) you have and also makes it easier to refactor production code as well as maintain those tests in the future as you add new functionality.

Answer 4

To answer your question about suggesting values for tests, you could possibly take a look at Agitar (http://www.agitar.com/). I remember having a presentation about it yonks ago and I'm sure it has changed (hopefully for the better) since then, but it might be what you're looking for. I, myself, would continue to go down the route of TDD and the other things I mentioned in my other answer, but if you're interested, here's a little part of what the Agitar site has to say:

"The AgitarOne product family helps you work safer, better, and smarter as you develop and maintain your Java applications. AgitarOne JUnit Generator creates thorough JUnit tests on your code. This helps you find regressions and makes it safer and easier to improve your code to reduce the cost to maintain it."