Retry a Visual Studio C# TestMethod

Question

I'm curious to know if there's any built-in mechanism to retry tests in the Visual Studio 2008 unit testing framework for C#. Case in point, I have a C# unit test which looks something like:

[TestMethod]
public void MyMethod() {
    DoSomething();
    Assert.Something();
}

Now, occasionally DoSomething() performs badly; in that case I would like to rerun the DoSomething() method before reaching the assertion. Obviously I can do something like:

...
do {
    Initialization();
    DoSomething();
} while (PerformedOK() == false);
Assert.Something();
...

Though this is a bit cumbersome because of the added loop and repeating the test initialization which would otherwise be completely handled by other methods / class constructor.

My question is whether there is a more convenient mechanism for retrying a test, something like:

DoSomething();
if (PerformedOK() == false) Retry();
else Assert.Something();

which will automatically retry the test without registering it as a failure, while performing all the regular initialization code as usual.

Answer 1

Seriously...

"occasionally DoSomething() performs badly"

A test should be green every time. If the tested code sometimes perform "badly", then you need to fix your code, isolating the different behavior. You should have two test, one where it Asserts correct when DoSomething fails (and is supposed to fail), and one where it Asserts correct when DoSomething is ok (and is supposed to be ok).

Having retry logic in a test is just wrong imo. You should always Assert on the expected outcome, and you should be able to isolate and instrument your code to return what you expect.

[Edit - added some code which could be used for a retry loop]

You could create a loop wrapper which takes whatever method in and calls it X number of times, or until it succeeds. You could also have the Loop function call your init, or pass it as a separate argument. The method could also return a bool if successful. Change the signature to fit your needs.

[TestMethod]
public void something()
{
   Loop.LoopMe(TestMethod,3);            
   Assert.Something();
}

class Loop
{
    public static void LoopMe(Action action, int maxRetry )
    {
        Exception lastException = null;
        while (maxRetry > 0)
        {
            try
            {
                action();
                return;
            }
            catch (Exception e)
            {
                lastException = e;
                maxRetry--;                    
            }                
        }
        throw lastException;
    }

}

Answer 2

Your second example is almost the same lines of code and same complexity. There are tons of ways to skin it, you could not that I am advocating it use recursion.

[TestMethod]
public void MyMethod() {
   bool success = DoSomething();
   Assert.IsTrue(success);
}

public boolean DoSomething(){
    //Do whatever

    if(performedOk){
       return true;
    }else{
        //find a way to stop it.
    }

}

But the point is it is a unit test. If something is causing it to go wrong, you need to find a way isolate your test, so that it is in a controlled environment.

Unless you have a requirement that says, test should pass eventually. The best retry logic you should use, is after it fails. Click the test and hit run again.