How do I enforce exception message with ExpectedException attribute

Question

I thought these two tests should behave identically, in fact I have written the test in my project using MS Test only to find out now that it does not respect the expected message in the same way that NUnit does.

NUnit (fails):

[Test, ExpectedException(typeof(System.FormatException), ExpectedMessage = "blah")]
public void Validate()
{
    int.Parse("dfd");
}

MS Test (passes):

[TestMethod, ExpectedException(typeof(System.FormatException), "blah")]
public void Validate()
{
    int.Parse("dfd");
}

No matter what message I give the ms test, it will pass.

Is there any way to get the ms test to fail if the message is not right? Can I even create my own exception attribute? I would rather not have to write a try catch block for every test where this occurs.

Brummell

Answer 1

That mstest second parameter is a message that is printed out when the test fails. The mstest will succeed if a formatexception is thrown. I found this post that may be useful

http://blogs.msdn.com/b/csell/archive/2006/01/13/expectedexception-might-not-be-what-you-ve-expected.aspx

Hundredth

Thanks, thats some great information. Do you know a way using ms test to write a test that behaves the same as the nunit one? – Brummell

I might look at creating a custom attribute like the nunit one. – Brummell

The link is dead. – Trawl

the link is dead – Nonparticipation

Answer 2

We use this attribute all over the place and we clearly misunderstood the second parameter (shame on us).

However, we definitely have used it to check the exception message. The following was what we used with hints from this page. It doesn't handle globalization, or inherited exception types, but it does what we need. Again, the goal was to simply RR 'ExpectedException' and swap it out with this class. (Bummer ExpectedException is sealed.)

public class ExpectedExceptionWithMessageAttribute : ExpectedExceptionBaseAttribute
{
    public Type ExceptionType { get; set; }

    public string ExpectedMessage { get; set; }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType)
    {
        this.ExceptionType = exceptionType;
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage)
    {
        this.ExceptionType = exceptionType;
        this.ExpectedMessage = expectedMessage;
    }

    protected override void Verify(Exception e)
    {
        if (e.GetType() != this.ExceptionType)
        {
            Assert.Fail($"ExpectedExceptionWithMessageAttribute failed. Expected exception type: {this.ExceptionType.FullName}. " +
                $"Actual exception type: {e.GetType().FullName}. Exception message: {e.Message}");
        }

        var actualMessage = e.Message.Trim();
        if (this.ExpectedMessage != null)
        {
            Assert.AreEqual(this.ExpectedMessage, actualMessage);
        }

        Debug.WriteLine($"ExpectedExceptionWithMessageAttribute:{actualMessage}");
    }
}

Yod

I tried to achieve the same thing using PostSharp, but inheriting the existing attribute makes alot more sense! – Crocket

Literally just made this class myself. Though I called it PrintExpectedExceptionAttribute :) – Ankylose

Nice! As they say, 'Great minds think alike and do exactly the same work in parallel'. I gotta say, we use this attribute all the time still. – Yod

Thought I'd add that we now basically use a ProjectAssert (with 'Project' being our Project name) that has a static method: ExpectException(Action action,Type exceptionType, string message = null); So it could look like ProjectAssert.ExpectException(()=>{var x = 0; var y = 1/x;},typeof(DivideByZeroException)). It's nice getting it inline but basically does the exact same thing the attribute above does. – Yod

Answer 3

@rcravens is correct - the second param is a message that is printed if the test fails. What I've done to work around this is craft my tests a little differently. Admittedly, I don't love this approach, but it works.

[TestMethod]
public void Validate()
{
    try
    {
        int.Parse("dfd");
    
        // Test fails if it makes it this far
        Assert.Fail("Expected exception was not thrown.");
    }
    catch (ArgumentNullException ex) // <-- Expected Exception class
    {
        Assert.AreEqual("blah", ex.Message);
    }
    catch (Exception ex) // All the other exceptions
    {
        Assert.Fail("Thrown exception was of the wrong type");
    }
}

Strangulate

I am a fan of this approach now if using mstest – Brummell

Thanks, this will quickly do what I want it to do, validate my message – Thomasson

This is actually pretty crafty workaround.. I love it! – Consolute

This would validate ONLY the message. If you also want to validate the exception type, the catch would have to be something like catch (ArgumentNullException ex){ Assert.AreEqual("blah", ex.Message); } catch (Exception ex){ Assert.Fail("Thrown exception was of the wrong type"); } – Selfabsorbed

Answer 4

I extended answer from BlackjacketMack for our project by adding support for contains, case-insensitive and ResourcesType-ResourceName combinations.

Usage example:

public class Foo
{
    public void Bar(string mode)
    {
        if (string.IsNullOrEmpty(mode)) throw new InvalidOperationException(Resources.InvalidModeSpecified);
    }

    public void Bar(int port)
    {
        if (port < 0 || port > Int16.MaxValue) throw new ArgumentOutOfRangeException("port");
    }
}

[TestClass]
class ExampleClass
{
    [TestMethod]
    [ExpectedExceptionWithMessage(typeof(InvalidOperationException), typeof(Samples.Resources), "InvalidModeSpecified")]
    public void Raise_Exception_With_Message_Resource()
    {
        new Foo().Bar(null);
    }

    [TestMethod]
    [ExpectedExceptionWithMessage(typeof(ArgumentOutOfRangeException), "port", true)]
    public void Raise_Exeception_Containing_String()
    {
        new Foo().Bar(-123);
    }
}

Here is the updated class:

public class ExpectedExceptionWithMessageAttribute : ExpectedExceptionBaseAttribute
{
    public Type ExceptionType { get; set; }

    public Type ResourcesType { get; set; }

    public string ResourceName { get; set; }

    public string ExpectedMessage { get; set; }

    public bool Containing { get; set; }

    public bool IgnoreCase { get; set; }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType)
        : this(exceptionType, null)
    {
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage)
        : this(exceptionType, expectedMessage, false)
    {
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, string expectedMessage, bool containing)
    {
        this.ExceptionType = exceptionType;
        this.ExpectedMessage = expectedMessage;
        this.Containing = containing;
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, Type resourcesType, string resourceName)
        : this(exceptionType, resourcesType, resourceName, false)
    {
    }

    public ExpectedExceptionWithMessageAttribute(Type exceptionType, Type resourcesType, string resourceName, bool containing)
    {
        this.ExceptionType = exceptionType;
        this.ExpectedMessage = ExpectedMessage;
        this.ResourcesType = resourcesType;
        this.ResourceName = resourceName;
        this.Containing = containing;
    }

    protected override void Verify(Exception e)
    {
        if (e.GetType() != this.ExceptionType)
        {
            Assert.Fail(String.Format(
                            "ExpectedExceptionWithMessageAttribute failed. Expected exception type: <{0}>. Actual exception type: <{1}>. Exception message: <{2}>",
                            this.ExceptionType.FullName,
                            e.GetType().FullName,
                            e.Message
                            )
                        );
        }

        var actualMessage = e.Message.Trim();

        var expectedMessage = this.ExpectedMessage;

        if (expectedMessage == null)
        {
            if (this.ResourcesType != null && this.ResourceName != null)
            {
                PropertyInfo resourceProperty = this.ResourcesType.GetProperty(this.ResourceName, BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public);
                if (resourceProperty != null)
                {
                    string resourceValue = null;

                    try
                    {
                        resourceValue = resourceProperty.GetMethod.Invoke(null, null) as string;
                    }
                    finally
                    {
                        if (resourceValue != null)
                        {
                            expectedMessage = resourceValue;
                        }
                        else
                        {
                            Assert.Fail("ExpectedExceptionWithMessageAttribute failed. Could not get resource value. ResourceName: <{0}> ResourcesType<{1}>.",
                            this.ResourceName,
                            this.ResourcesType.FullName);
                        }
                    }
                }
                else
                {
                    Assert.Fail("ExpectedExceptionWithMessageAttribute failed. Could not find static resource property on resources type. ResourceName: <{0}> ResourcesType<{1}>.",
                        this.ResourceName,
                        this.ResourcesType.FullName);
                }
            }
            else
            {
                Assert.Fail("ExpectedExceptionWithMessageAttribute failed. Both ResourcesType and ResourceName must be specified.");
            }
        }

        if (expectedMessage != null)
        {
            StringComparison stringComparison = this.IgnoreCase ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal;
            if (this.Containing)
            {
                if (actualMessage == null || actualMessage.IndexOf(expectedMessage, stringComparison) == -1)
                {
                    Assert.Fail(String.Format(
                                    "ExpectedExceptionWithMessageAttribute failed. Expected message: <{0}>. Actual message: <{1}>. Exception type: <{2}>",
                                    expectedMessage,
                                    e.Message,
                                    e.GetType().FullName
                                    )
                                );
                }
            }
            else
            {
                if (!string.Equals(expectedMessage, actualMessage, stringComparison))
                {
                    Assert.Fail(String.Format(
                                    "ExpectedExceptionWithMessageAttribute failed. Expected message to contain: <{0}>. Actual message: <{1}>. Exception type: <{2}>",
                                    expectedMessage,
                                    e.Message,
                                    e.GetType().FullName
                                    )
                                );
                }
            }
        }
    }
}

Trompe

As explained here https://mcmap.net/q/450981/-mstest-extending-expectedexceptionbaseattribute-hides-test-failure-explanation, this does not work as expected in version 11 (VS2012) because it will always show "Exception has been thrown by the target of an invocation". It is fixed at least starting version 14. – Selfabsorbed

Answer 5

You can use code from this project explained in this article to create an ExpectedExceptionMessage attribute which will work with MS Test to get the desired result.

ms test (fails):

[TestClass]
public class Tests : MsTestExtensionsTestFixture
{
    [TestMethod, ExpectedExceptionMessage(typeof(System.FormatException), "blah")]
    public void Validate()
    {
        int.Parse("dfd");
    }
}

Brummell

Answer 6

Try this modern way:

[TestMethod]
public async Task DoSmthAsync_WithWrongParam_ThrowHttpRequestException()
{
    var smbdTask = service.DoSmthAsync(/* call params */);

    var exception = await Assert.ThrowsExceptionAsync<HttpRequestException>(() => smbdTask);
    Assert.AreEqual(HttpStatusCode.NotFound, exception.StatusCode);
}

Plainsman

Yes this question is ancient now. These days I use fluent assertions in the test method code and no attributes similar to your answer. – Brummell

Answer 7

I wrote this a while back, so maybe there is a better way in VS2012.

http://dripcode.blogspot.com.au/search?q=exception

Forklift

Answer 8

Try This. In This example I have a greeting message that displays the greeting message for a given name. When the name parameter is empty or null system throws an exception with message. The ExceptionAssert.Throws verify both in MsTest.

[TestMethod]
public void Does_Application_Display_Correct_Exception_Message_For_Empty_String()
{
    // Arrange
    var oHelloWorld = new HelloWorld();

    // Act

    // Asset

    ExceptionAssert.Throws<ArgumentException>(() => 
            oHelloWorld.GreetingMessge(""),"Invalid Name, Name can't be empty");
}

[TestMethod]
public void Does_Application_Display_Correct_Exception_Message_For_Null_String()
{
    // Arrange
    var oHelloWorld = new HelloWorld();

    // Act

    // Asset

    ExceptionAssert.Throws<ArgumentNullException>(() => 
        oHelloWorld.GreetingMessge(null), "Invalid Name, Name can't be null");
}

Profundity

ExceptionAssert doesn't seems to be bundled in Microsoft UnitTesting, are you using a third party plugin? – Unaccustomed