Can I repeat a unit test repeatedly over a collection in Visual Studio 2008?

Question

I have a unit test where I run a method for every item in a collection (using a foreach). Currently I have an assert at the end of the foreach to test if the method returned the correct value. This works, but the result is the test fails when the method fails for the first time. No subsiquent items in the collection are tested.

I would like to essentially run a full fledged test for each item in the collection. The test is run for every item, even if one fails. And the test results window shows result data for every pass.

I know I can use a datasource to run a test for each row in a database, but this is not convenient in this case. Is there another way?

My current test looks like:

public void TestXHTMLBlock()
        {
            foreach (XmlNode current in Test_Cases.SelectNodes("Test_Cases/Test_Case"))
            {
                XHTMLBlock x = new XHTMLBlock(current.SelectSingleNode("Input").CreateNavigator(), "");
                XmlDocument temp = new XmlDocument();
                temp.LoadXml("<Output>" + x.sWordML(false) + "</Output>");


                XmlDiff diff = new XmlDiff(XmlDiffOptions.IgnoreComments |
                                     XmlDiffOptions.IgnoreDtd |
                                     XmlDiffOptions.IgnoreNamespaces |
                                     XmlDiffOptions.IgnorePI |
                                     XmlDiffOptions.IgnorePrefixes |
                                     XmlDiffOptions.IgnoreWhitespace |
                                     XmlDiffOptions.IgnoreXmlDecl);

                StringBuilder str = new StringBuilder();
                XmlWriter xwrite = XmlWriter.Create(str);

                bool ret = diff.Compare(current.SelectSingleNode("Output"), temp.SelectSingleNode("Output"), xwrite);
                xwrite.Close();

                Assert.IsTrue(ret, current.SelectSingleNode("Description").InnerText);
            }
        }

I know this looks like a prime candidate for an XML datasource, but I can't get that to work. The Input tag contains XML which the datasource attempts to parse. This means I can't properly access it in my test and the datasource parser chokes on some of my input XML. If I were able to tell the datasource XML parser to treat the internal XML as text, that would work as well...

Answer 1

You can put the result of testing an individual item in a collection and assert after foreach if that collection contains anything. Below is sample code snippet to illustrate the idea.

List<Item> errors = new List<Item>();
foreach(Item item in items)
{
   if( string.IsNullOrEmpty(item.Name))
   {
     errors.Add(item);
   }
} 
Assert.IsTrue(errors.Count == 0,"There are items with no name");

If you want to see more details when the test fails, you can write a utility method that iterates through errors collection and generates a detailed message that you can then use in your assert.

Answer 2

If you're using nUnit 2.5.x, the ValueSourceAttribute may be what you are looking for. That will repeat the whole test once for each member of the indicated IEnumerable.

Answer 3

I would consider using a StringBuilder and appending a string on each error.

At the end you can check if the StringBuilder is empty and then write it to console/debug/trace, whatever your unit testing framework is automatically collecting and let the test fail via Assert.Fail() or similar