How can I test for an expected exception with a specific exception message from a resource file in Visual Studio Test?

Question

Visual Studio Test can check for expected exceptions using the ExpectedException attribute. You can pass in an exception like this:

[TestMethod]
[ExpectedException(typeof(CriticalException))]
public void GetOrganisation_MultipleOrganisations_ThrowsException()

You can also check for the message contained within the ExpectedException like this:

[TestMethod]
[ExpectedException(typeof(CriticalException), "An error occured")]
public void GetOrganisation_MultipleOrganisations_ThrowsException()

But when testing I18N applications I would use a resource file to get that error message (any may even decide to test the different localizations of the error message if I want to, but Visual Studio will not let me do this:

[TestMethod]
[ExpectedException(typeof(CriticalException), MyRes.MultipleOrganisationsNotAllowed)]
public void GetOrganisation_MultipleOrganisations_ThrowsException()

The compiler will give the following error:

An attribute argument must be a constant expression, typeof expression or array creation expression of an attribute

Does anybody know how to test for an exception that has a message from a resource file?

---

One option I have considered is using custom exception classes, but based on often heard advice such as:

"Do create and throw custom exceptions if you have an error condition that can be programmatically handled in a different way than any other existing exception. Otherwise, throw one of the existing exceptions." Source

I'm not expecting to handle the exceptions differently in normal flow (it's a critical exception, so I'm going into panic mode anyway) and I don't think creating an exception for each test case is the right thing to do. Any opinions?

Deeply

The message parameter does not get checked against the message of the thrown exception. – Vargueno

Answer 1

Just an opinion, but I would say the error text:

• is part of the test, in which case getting it from the resource would be 'wrong' (otherwise you could end up with a consistantly mangled resource), so just update the test when you change the resource (or the test fails)
• is not part of the test, and you should only care that it throws the exception.

Note that the first option should let you test multiple languages, given the ability to run with a locale.

As for multiple exceptions, I'm from C++ land, where creating loads and loads of exceptions (to the point of one per 'throw' statement!) in big heirachies is acceptable (if not common), but .Net's metadata system probably doesn't like that, hence that advice.

Mirnamirror

I would implement a simple proc that reads the resource file and finds the message to be written, your method is a great way to de-map and lose valuable time updating tests (what we are most trying to avoid) – Demb

@Mickey: Well then, you're in luck: the other answer is what you are looking for! If you specifically want a unit-test that it retrieves the same message, that's fine, but I tend to prefer that it retrieves the right message. But if your project is different, then you can easily have a different "right" answer! – Mirnamirror

I don't follow, using DataBound Unit testing, you can keep your cake and eat it too. map the resources in the databind and map your message culture in the test, or vice versa. no ? – Demb

@Mickey: Yes, you can, but my original post was pointing out that you then don't catch messages defined incorrectly in the first place: my position was that when testing, you check the result against explicit values, not something that should give the same result: for a method that does 2 + 2, you test it returns 4, not 2 + 2. If you care what the message is, then I think you should clearly check it gives the right result, not just the same result. But feel free to do things differently if you have reason to! – Mirnamirror

I did not actually understand that until now. In that case, I agree – Demb

Answer 2

I would recommend using a helper method instead of an attribute. Something like this:

public static class ExceptionAssert
{
  public static T Throws<T>(Action action) where T : Exception
  {
    try
    {
      action();
    }
    catch (T ex)
    {
      return ex;
    }
    Assert.Fail("Exception of type {0} should be thrown.", typeof(T));

    //  The compiler doesn't know that Assert.Fail
    //  will always throw an exception
    return null;
  }
}

Then you can write your test something like this:

[TestMethod]
public void GetOrganisation_MultipleOrganisations_ThrowsException()
{
  OrganizationList organizations = new Organizations();
  organizations.Add(new Organization());
  organizations.Add(new Organization());

  var ex = ExceptionAssert.Throws<CriticalException>(
              () => organizations.GetOrganization());
  Assert.AreEqual(MyRes.MultipleOrganisationsNotAllowed, ex.Message);
}

This also has the benefit that it verifies that the exception is thrown on the line you were expecting it to be thrown instead of anywhere in your test method.

Raila

Answer 3

The ExpectedException Message argument does not match against the message of the exception. Rather this is the message that is printed in the test results if the expected exception did not in fact occur.

Umlaut

Answer 4

I think you can just do an explicit try-catch in your test code instead of relying on the ExpectedException attribute to do it for you. Then you can come up with some helper method that will read the resource file and compare the error message to the one that comes with the exception that was caught. (of course if there wasn't an exception then the test case should be considered a fail)

Valoniah

Answer 5

If you switch over to using the very nice xUnit.Net testing library, you can replace [ExpectedException] with something like this:

[Fact]
public void TestException()
{
   Exception ex = Record.Exception(() => myClass.DoSomethingExceptional());
   // Assert whatever you like about the exception here.
}

Holography

Answer 6

I wonder if NUnit is moving down the path away from simplicity... but here you go.

New enhancements (2.4.3 and up?) to the ExpectedException attribute allow you more control on the checks to be performed on the expected Exception via a Handler method. More Details on the official NUnit doc page.. towards the end of the page.

[ExpectedException( Handler="HandlerMethod" )]
public void TestMethod()
{
...
}

public void HandlerMethod( System.Exception ex )
{
...
}

Note: Something doesn't feel right here.. Why are your exceptions messages internationalized.. Are you using exceptions for things that need to be handled or notified to the user. Unless you have a bunch of culturally diverse developers fixing bugs.. you shouldn't be needing this. Exceptions in English or a common accepted language would suffice. But in case you have to have this.. its possible :)

Geneticist

You don't think the method of getting the error dialog message from the throwen exception is good design? I was under the impression it was common, and it simplifes error path code... I know I wouldn't want to see a dialog in Chinese! – Mirnamirror

I'm using NUnit framework version 2.6 and the attribute [ExpectedException( Handler="HandlerMethod" )] does not compile – Schizothymia

@mishrsud - I believe the compiler will give you more details on that. Check the output pane - what is it complaining about ? – Geneticist

@Geneticist the compiler complains about "unknown symbol" so it doesn't seem to know this particular signature of the attribute – Schizothymia

@mishrsud- the example is straight out of the documentation page of nunit v2.6.2 nunit.org/index.php?p=exception&r=2.6.2 | can you check if you are not picking up the wrong version of the assembly, have the necessary using stmts,clean the project ? Just thinking aloud here... – Geneticist

Answer 7

I came across this question while trying to resolve a similar issue on my own. (I'll detail the solution that I settled on below.)

I have to agree with Gishu's comments about internationalizing the exception messages being a code smell.

I had done this initially in my own project so that I could have consistency between the error messages throw by my application and in my unit tests. ie, to only have to define my exception messages in one place and at the time, the Resource file seemed like a sensible place to do this since I was already using it for various labels and strings (and since it made sense to add a reference to it in my test code to verify that those same labels showed in the appropriate places).

At one point I had considered (and tested) using try/catch blocks to avoid the requirement of a constant by the ExpectedException attribute, but this seemed like it would lead to quite a lot of extra code if applied on a large scale.

In the end, the solution that I settled on was to create a static class in my Resource library and store my exception messages in that. This way there's no need to internationalize them (which I'll agree doesn't make sense) and they're made accessible anytime that a resource string would be accessible since they're in the same namespace. (This fits with my desire not to make verifying the exception text a complex process.)

My test code then simply boils down to (pardon the mangling...):

[Test, 
    ExpectedException(typeof(System.ArgumentException),
    ExpectedException=ProductExceptionMessages.DuplicateProductName)]
public void TestCreateDuplicateProduct()
{
    _repository.CreateProduct("TestCreateDuplicateProduct");
    _repository.CreateProduct("TestCreateDuplicateProduct");
} 

Animadvert