Multiple asserts in single test?

Question

Let's say I want to write a function that validates an email address with a regex. I write a little test to check my function and write the actual function. Make it pass.

However, I can come up with a bunch of different ways to test the same function ([email protected]; [email protected]; test.test.com, etc).

Do I put all the incantations that I need to check in the same, single test with several ASSERTS or do I write a new test for every single thing I can think of?

Thanks!

Answer 1

Most testing frameworks now support some sort of data based testing to let you run the same test on multiple data sets.

See the ValuesAttribute in NUnit.

xUnit.net, MBUnit and others have similar methods.

Answer 2

I don't think that you should write separate test for each case since all the cases are related to the same thing which is testing that the string is the correct email address. If you are using MbUnit or NUnit for running your tests then you can use the RowTest and Row attribute to pass in different values to the test. Another way is to store all the different email formats in an array and loop over the array and perform the assert.

Answer 3

If the different incantations really boil down to the same thing it isn't a problem.

However, if one of the email addresses suddenly breaks the test, you are going to have to debug to find out which case went wrong. So that seems a good reason to break them up.

You would end up with millions of unit tests, which is right as they are after all tesing a unit of your application, but in practice multiple asserts will do providing that the way in which a failed assert breaks the test doesn't mess up the meaning of the failure.

Answer 4

If you are testing the same functionality, I'd check them in the same test, with multiple assertions, provided that your tests remain simple enough.

Some people advocate that every test must have just a single assertion, because the test must be very, very, very simple.

If the test is not simple, say that you have loops and ifs, you would need a test for the test itself, to check that its logic is correct, and this is not good.

If your test has multiple assertions, but still remain simple (no loops, no ifs) and the multiple assertions are testing the same thing, then I'd not be so aggressive in advocating "single assert per test". The choice is yours.

Answer 5

Generally I create many different tests, and give each it's own name. For example let's say that there are 3 different regexes {A, B, & C} for matching an email address. The function checks the incoming email for a match and accepts the first match found.

I would have the following tests.

• ATypeEmailShouldMatchPatternA
• BTypeEmailShouldMatchPatternB
• CTypeEmailShouldMatchPatternC
• BadEmailShouldNotMatchAnyPattern
• EmailCompatibleWithPatternAAndPatternBShouldBeMatchedByA
• EmailCompatibleWithPatternAAndCShouldBeMatchedByA
• EmailCompatibleWithPatternBAndCShouldBeMatchedByB

Generally I'd put one assert in each test.

Sometimes, however, I'll put more than one assert in a test if the asserts are all checking different parts of the same thing. For example, let's say I have a function that finds all the emails matching pattern A. My test feeds in a list of emails but only one matches pattern A. The function returns a list, and I expect that list to have only one element in it. So I would assert two things:

1. That the list has one element in it.
2. That the one element is the email that matches pattern A.

Answer 6

As @Paul mentioned several test frameworks support RowTests. Using that feature you can write something as monstrous as this:

[TestCase ("[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("x#[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("x%[email protected]", true)]
[TestCase ("x&[email protected]", true)]
[TestCase ("x'[email protected]", true)]
[TestCase ("x*[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("x/[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("x^[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("x`[email protected]", true)]
[TestCase ("x{[email protected]", true)]
[TestCase ("x{[email protected]", true)]
[TestCase ("x|[email protected]", true)]
[TestCase ("x}[email protected]", true)]
[TestCase ("[email protected]", true)]
[TestCase ("test", false)]
[TestCase ("", false)]
[TestCase (null, false)]
public void IsEmail_Should_Match_Valid_Email_Addresses(string target, bool result)
{
    Assert.AreEqual(result, target.IsEmail());
}

Or you could do the same with a bunch of asserts. It's common to assert multiple properties on an object after performing some action. I think the above solution is more readable though.

Answer 7

BDD and/or Context/Specification frameworks like SubSpec which happens to have some salient examples manage this by treating each batch of related assertions as a separate Observation block that one gives a name or descriptive label to.

The key elements of a good test that this pushes one towards are:

1. you have AAA / GWT separation of each of the phases - you're forced to really think about which is which [and will often quickly realise when things need to be broken up more]
2. You end up with good names/descriptions on Observations (Row Tests generally leave you with [less maintainable] comments at best)
3. When tests 10 and 17 out of 24 [as with Row Tests] you have the status of each of the tests to assist you in narrowing down the solution

Row Tests can be appropriate in some cases when you really are doing a simple table or matrix based suite of things.

Also xUnit.net's PropertyData can be powerful and appropriate in some cases as a way of doing some of the tricks in the other anwers.