How can I test for an expected exception with a specific exception message from a resource file in Visual Studio Test?

Question

Visual Studio Test can check for expected exceptions using the ExpectedException attribute. You can pass in an exception like this:

[TestMethod]
[ExpectedException(typeof(CriticalException))]
public void GetOrganisation_MultipleOrganisations_ThrowsException()

You can also check for the message contained within the ExpectedException like this:

[TestMethod]
[ExpectedException(typeof(CriticalException), "An error occured")]
public void GetOrganisation_MultipleOrganisations_ThrowsException()

But when testing I18N applications I would use a resource file to get that error message (any may even decide to test the different localizations of the error message if I want to, but Visual Studio will not let me do this:

[TestMethod]
[ExpectedException(typeof(CriticalException), MyRes.MultipleOrganisationsNotAllowed)]
public void GetOrganisation_MultipleOrganisations_ThrowsException()

The compiler will give the following error:

An attribute argument must be a constant expression, typeof expression or array creation expression of an attribute

Does anybody know how to test for an exception that has a message from a resource file?

---

One option I have considered is using custom exception classes, but based on often heard advice such as:

"Do create and throw custom exceptions if you have an error condition that can be programmatically handled in a different way than any other existing exception. Otherwise, throw one of the existing exceptions." Source

I'm not expecting to handle the exceptions differently in normal flow (it's a critical exception, so I'm going into panic mode anyway) and I don't think creating an exception for each test case is the right thing to do. Any opinions?

Answer 1

I think you can just do an explicit try-catch in your test code instead of relying on the ExpectedException attribute to do it for you. Then you can come up with some helper method that will read the resource file and compare the error message to the one that comes with the exception that was caught. (of course if there wasn't an exception then the test case should be considered a fail)

Answer 2

If you switch over to using the very nice xUnit.Net testing library, you can replace [ExpectedException] with something like this:

[Fact]
public void TestException()
{
   Exception ex = Record.Exception(() => myClass.DoSomethingExceptional());
   // Assert whatever you like about the exception here.
}

Answer 3

Just an opinion, but I would say the error text:

• is part of the test, in which case getting it from the resource would be 'wrong' (otherwise you could end up with a consistantly mangled resource), so just update the test when you change the resource (or the test fails)
• is not part of the test, and you should only care that it throws the exception.

Note that the first option should let you test multiple languages, given the ability to run with a locale.

As for multiple exceptions, I'm from C++ land, where creating loads and loads of exceptions (to the point of one per 'throw' statement!) in big heirachies is acceptable (if not common), but .Net's metadata system probably doesn't like that, hence that advice.

Answer 4

I wonder if NUnit is moving down the path away from simplicity... but here you go.

New enhancements (2.4.3 and up?) to the ExpectedException attribute allow you more control on the checks to be performed on the expected Exception via a Handler method. More Details on the official NUnit doc page.. towards the end of the page.

[ExpectedException( Handler="HandlerMethod" )]
public void TestMethod()
{
...
}

public void HandlerMethod( System.Exception ex )
{
...
}

Note: Something doesn't feel right here.. Why are your exceptions messages internationalized.. Are you using exceptions for things that need to be handled or notified to the user. Unless you have a bunch of culturally diverse developers fixing bugs.. you shouldn't be needing this. Exceptions in English or a common accepted language would suffice. But in case you have to have this.. its possible :)

Answer 5

I would recommend using a helper method instead of an attribute. Something like this:

public static class ExceptionAssert
{
  public static T Throws<T>(Action action) where T : Exception
  {
    try
    {
      action();
    }
    catch (T ex)
    {
      return ex;
    }
    Assert.Fail("Exception of type {0} should be thrown.", typeof(T));

    //  The compiler doesn't know that Assert.Fail
    //  will always throw an exception
    return null;
  }
}

Then you can write your test something like this:

[TestMethod]
public void GetOrganisation_MultipleOrganisations_ThrowsException()
{
  OrganizationList organizations = new Organizations();
  organizations.Add(new Organization());
  organizations.Add(new Organization());

  var ex = ExceptionAssert.Throws<CriticalException>(
              () => organizations.GetOrganization());
  Assert.AreEqual(MyRes.MultipleOrganisationsNotAllowed, ex.Message);
}

This also has the benefit that it verifies that the exception is thrown on the line you were expecting it to be thrown instead of anywhere in your test method.

Answer 6

The ExpectedException Message argument does not match against the message of the exception. Rather this is the message that is printed in the test results if the expected exception did not in fact occur.

Answer 7

I came across this question while trying to resolve a similar issue on my own. (I'll detail the solution that I settled on below.)

I have to agree with Gishu's comments about internationalizing the exception messages being a code smell.

I had done this initially in my own project so that I could have consistency between the error messages throw by my application and in my unit tests. ie, to only have to define my exception messages in one place and at the time, the Resource file seemed like a sensible place to do this since I was already using it for various labels and strings (and since it made sense to add a reference to it in my test code to verify that those same labels showed in the appropriate places).

At one point I had considered (and tested) using try/catch blocks to avoid the requirement of a constant by the ExpectedException attribute, but this seemed like it would lead to quite a lot of extra code if applied on a large scale.

In the end, the solution that I settled on was to create a static class in my Resource library and store my exception messages in that. This way there's no need to internationalize them (which I'll agree doesn't make sense) and they're made accessible anytime that a resource string would be accessible since they're in the same namespace. (This fits with my desire not to make verifying the exception text a complex process.)

My test code then simply boils down to (pardon the mangling...):

[Test, ExpectedException(typeof(System.ArgumentException), ExpectedException=ProductExceptionMessages.DuplicateProductName)] public void TestCreateDuplicateProduct() { _repository.CreateProduct("TestCreateDuplicateProduct"); _repository.CreateProduct("TestCreateDuplicateProduct"); }