How to implement XUnit descriptive Assert message?

Question

Context

in XUnit github I found this: Add Assert.Equal(expected, actual, message) overload #350 (so a developer ask for a non existing overload see below)

Quote from the answer:

We are a believer in self-documenting code; that includes your assertions.

(so the XUnit team rejects it)

OK, I got it. I also believe the self documenting code. Still I can not find out this use case:

Sample

// Arrange
// Create some external soap service client and its wrapper classes

// Act
// client.SomeMethod();

// Assert
// Sorry, soap service's interface, behaviour and design is *given*
// So I have to check if there is no Error, and 
// conveniently if there is, then I would like to see it in the assertion message

Assert.Equal(0, client.ErrorMessage.Length); // Means no error

// I would like to have the same result what would be the following *N*U*n*i*t* assert:
// Assert.AreEqual(0, client.ErrorMessage.Length, client.ErrorMessage); // Means no error

Question

How can I implement a descriptive assert message in this case in XUnit which still has no such an overload?

Patrology

I'm unclear on what the issue is. Why are you not just using Assert.AreEqual(0, client.ErrorMessage.Length, client.ErrorMessage); as you pointed out in the comment? – Gilmer

There is no such overload in XUnit. That's an NUnit call. Please see the very starting sentence: a developer asks for such an overload, and XUnit team rejects because of the quoted "We are a believer in self-documenting code; that includes your assertions" – Patrology

@g.pickardou, Why not use the suggestions provided at the link. Like fluent assertions or create your own assertion that wraps the Assert.True or Assert.False which were left with their message overloads. It was mentioned further down You can provide messages to Assert.True and .False. If you simply cannot live without messages (and refuse to use a different assertion), you could always fall back to: Assert.True(number == 2, "This is my message"); – Drexler

@Nikosi: Because I did not get that :-). That's an answer, however I still not find/get the fluent sample you are referring. – Patrology

@Patrology Fluent assertions is another library – Drexler

It's... let's say 'amusing', that the XUnit maintainers locked the ticket you referenced, to ensure they wouldn't have to hear any more votes for this feature (after saying they'd made up their minds). The case for it is clear: emitting test state upon failure. Code can obviously be self-documenting and still benefit from emitting output, because output does not have to be hardcoded as per the XUnit assumptions. Assert.True("All output must be" == "hardcoded") failed. I just started to transition libraries, and would have dealt with the shortcoming; but not that attitude. – Jones

@Jones Indeed. If Microsoft were to remove the comment feature from C# and say "We believe in self-documenting code, so get gud." it would not be much different. It is difficult for me to accept their rationale. There is value in information, and as you point out, only certain, limited types of information can be embedded in static code and/or comments. We can resort to loggers, but it sure would be nice to keep our testing tech and workflow simple and integrated. – Grower

Answer 1

Use the suggestions provided at the link. Like fluent assertions or create your own assertion that wraps the Assert.True or Assert.False which were left with their message overloads. It was mentioned further down

Quote

You can provide messages to Assert.True and .False. If you simply cannot live without messages (and refuse to use a different assertion), you could always fall back to:

Assert.True(number == 2, "This is my message");

Quote:

If you really want to have messages you could add Fluent Assertions or maybe xbehave to your test projects and use their syntax. Fluent Assertions even throws xunit.net exceptions if it encounters its presence.

Drexler

That's an answer, however I still not find/get the fluent sample you are referring in your comment – Patrology

It took time, but finally I got it. (It's the zillions unit test framework I have to pick up and instantly work with...) – Patrology

Fluent assertions is very easy to pick up. I use it a lot and the learning curve was not steep. – Drexler

falling back to Assert.True(number == 2, "This is my message"); would result in an even fuzzier message that hides the actual values that were compared: "This is my message <newline> Expected: True <newline> Actual: False" so that's not really an option (I'm not complaining on you @Nkosi, of course, just thought it should be pointed out :) ) – Tenor

@Tenor you could include the values in your message: Assert.True(number == 2, $"This is my message\nExpected: 2\nActual: {number}" – Denominative

@Denominative while your suggestion does improve things a bit, you'd still get a longer message with 2 sets of expected/actual pairs of values: "This is my message <newline> Expected: 2 <newline> Actual: 1 <newline> Expected: True <newline> Actual: False". Thanks for the suggestion though :) Giving all the respect to Xunit, really, yet however opinionated they wish to be, IMHO self-documenting code has nothing to do with unit-test outputs on our CI/CD pipeline logs... :) – Tenor

This is rather a workaround than a solution or even a replacement. By using fluent-validations (which is bad anyway) you loose all the nice expected/actual hints in errors. – Carree

Answer 2

I was having the same issue. I've a test that pulls data from two web api's and then compares and asserts various things about the content. I started using standard XUnit assertions like:

Assert.Equal(HttpStatusCode.OK, response1.StatusCode);
Assert.Equal(HttpStatusCode.OK, response2.StatusCode);

But whilst this gives a useful message that a 404 has been returned, it not clear from the logs on our build/CI server which service caused the error message.

I ended up adding my own assertion to give context:

public class MyEqualException : Xunit.Sdk.EqualException
{
    public MyEqualException(object expected, object actual, string userMessage)
        : base(expected, actual)
    {
        UserMessage = userMessage;
    }

    public override string Message => UserMessage + "\n" + base.Message;
}

public static class AssertX
{
    /// <summary>
    /// Verifies that two objects are equal, using a default comparer.
    /// </summary>
    /// <typeparam name="T">The type of the objects to be compared</typeparam>
    /// <param name="expected">The expected value</param>
    /// <param name="actual">The value to be compared against</param>
    /// <param name="userMessage">Message to show in the error</param>
    /// <exception cref="MyEqualException">Thrown when the objects are not equal</exception>
    public static void Equal<T>(T expected, T actual, string userMessage)
    {
        bool areEqual;

        if (expected == null || actual == null)
        {
            // If either null, equal only if both null
            areEqual = (expected == null && actual == null);
        }
        else
        {
            // expected is not null - so safe to call .Equals()
            areEqual = expected.Equals(actual);
        }

        if (!areEqual)
        {
            throw new MyEqualException(expected, actual, userMessage);
        }
    }
}

Then I can do the same assertions as:

AssertX.Equal(HttpStatusCode.OK, response1.StatusCode, $"Fetching {Uri1}");
AssertX.Equal(HttpStatusCode.OK, response2.StatusCode, $"Fetching {Uri2}");

and the error log gives the actual,expected and prepends my message about which webapi was the culprit.

I realise I'm late to answer, but figured this might help others searching for a practical solution that don't have time to install/learn yet another test framework just to get useful information out of test failures.

Osmo

The custom Equal<T> method throws EqualException instead of your extended MyEqualException – Bake

I am starting wondering if I did well to pick xUnit instead of MSTest. The later offers much better assert options. Any thoughts on that? – Fare

Answer 3

Using a try/catch was enough for my purposes:

try
{
    Assert.Equal(expectedErrorCount, result.Count);
}
catch (EqualException ex)
{
    throw new XunitException($"{testMsg}\n{ex}");
}

Sainted

I believe this is the best answer; although I prefer and use FluentAssertions. – Underskirt

Answer 4

I stumbled upon the same issue and was surprised even 6 years later no one followed the suggestion to write custom assert methods. So I wrote one myself here.

Just add the nuget package and alias the AssertM class like this:

using Assert = XunitAssertMessages.AssertM;

all prior xunit assert methods are available so current asserts will continue to compile but have an added optional message parameter.

// This will work
Assert.Equal(0, client.ErrorMessage.Length)
// and so will this
Assert.Equal(0, client.ErrorMessage.Length, "Unexpected length")

Recor

Couple things to note. (1) CS8602 should consider Assert.NotNull in unit tests, (2) AssertM.Equal("expected", "actual") is ambiguous. AWESOME! This package is SOOO helpful. – Boarding

@DougDomeny Thanks for the awesome feedback. Both issues should be fixed in the latest 2.4.1 release. – Recor

The "ambiguous" error is still occurring. But I did confirm the first issue is fixed. Thanks for the quick response. Example, AssertM.Equal(paramType.Name, actualType.Name); (where Name is string) Error: CS0121 The call is ambiguous between the following methods or properties: 'AssertM.Equal<T>(T, T, string?)' and 'AssertM.Equal(string?, string?, bool, bool, bool, string?)' It's not a blocker because I has a failure message, but it's not strictly compatible. – Boarding

@DougDomeny Whoaw, thanks again! I totally missed that one. Should be fixed now in 2.4.2 – Recor