How can I have custom asserts with Shouldly and maintain the call-site-specific assertion messages?

Question

I'm using the excellent Shouldly library in my xUnit tests and I'm finding myself using the set sequence of assertions in different tests, so I'm combining them into new assertion extension methods - but when I do this I lose Shouldly's contextual assertion messages.

Here's my old code, which works with Shouldly to include source-level information and call-site context in the Shouldly assertion error:

[Fact]
public void Dict_should_contain_valid_Foobar_Bar_entry()
{
    IDictionary<String,Bar> dict = ...
    dict.TryGetValue( "Foobar", out Bar bar ).ShouldBeTrue();
    bar.ShouldNotBeNull();
    bar.ChildList.Count.ShouldBe( expected: 3 );
    bar.Message.ShouldBeNull();
}

[Fact]
public void Dict_should_contain_valid_Barbaz_Bar_entry()
{
    IDictionary<String,Bar> dict = ...
    dict.TryGetValue( "Barbaz", out Bar bar ).ShouldBeTrue();
    bar.ShouldNotBeNull();
    bar.ChildList.Count.ShouldBe( expected: 3 );
    bar.Message.ShouldBeNull();
}

I converted it to this new extension method in the same project:

public static void ShouldBeValidBar( this IDictionary<String,Bar> dict, String barName )
{
    dict.ShouldNotBeNull();
    dict.TryGetValue( barName, out Bar bar ).ShouldBeTrue();
    bar.ShouldNotBeNull();
    bar.ChildList.Count.ShouldBe( expected: 3 );
    bar.Message.ShouldBeNull();
}

And so my tests are changed to this:

[Fact]
public void Dict_should_contain_valid_Foobar_Bar_entry()
{
    IDictionary<String,Bar> dict = ...
    dict.ShouldBeValidBar( "Foobar" );
}

[Fact]
public void Dict_should_contain_valid_Barbaz_Bar_entry()
{
    IDictionary<String,Bar> dict = ...
    dict.ShouldBeValidBar( "Barbaz" );
}

...but now my Shouldly assert messages don't contain any contextual information from Dict_should_contain_valid_Foobar_Bar_entry and instead only contains context from ShouldBeValidBar.

How can I instruct Shouldly to disregard the context of ShouldBeValidBar and to use its parent call-site instead?

Mundane

Answer 1

TL;DR:

Add the [ShouldlyMethods] attribute to your custom assert extension method classes (not individual extension methods):

[ShouldlyMethods] // <-- This, right here!
public static class MyShouldlyAssertions
{
    public static void ShouldBeValidBar( this IDictionary<String,Bar> dict, String barName )
    {
        [...]
    }
}

Long version:

After some googling, and reading articles about how Shouldly works - and after reading the source of Shouldly's secret-sauce: SourceCodeTextGetter, I see that it determines which entries in the stack-trace can be overlooked by the presence of the [ShouldlyMethods] attribute (Shouldly.ShouldlyMethodsAttribute) on the method's containing type in each frame of the stack-trace:

void ParseStackTrace(StackTrace trace)
{
    [...]

    while (ShouldlyFrame == null || currentFrame.GetMethod().IsShouldlyMethod())
    {
        if (currentFrame.GetMethod().IsShouldlyMethod())
            ShouldlyFrame = currentFrame;

        [...]
    }

    [...]
}

internal static bool IsShouldlyMethod(this MethodBase method)
{
    if (method.DeclaringType == null)
        return false;

    return
        method
            .DeclaringType
            .GetCustomAttributes( typeof(ShouldlyMethodsAttribute), true )
            .Any()
        ||
        (
            method.DeclaringType.DeclaringType != null
            && 
            method
                .DeclaringType
                .DeclaringType
                .GetCustomAttributes( typeof(ShouldlyMethodsAttribute), true )
                .Any()
        );
}

So it's just a matter of adding the [ShouldlyMethods] attribute to my extension methods' container classes:

[ShouldlyMethods]
public static class ShouldlyAssertionExtensions
{    
    public static void ShouldBeValidBar( this IDictionary<String,Bar> dict, String barName )
    {
        dict.ShouldNotBeNull();
        dict.TryGetValue( barName, out Bar bar ).ShouldBeTrue();
        bar.ShouldNotBeNull();
        bar.ChildList.Count.ShouldBe( expected: 3 );
        bar.Message.ShouldBeNull();
    }
}

And now my assert errors have the context of the call-site of ShouldBeValidBar. Hurrah!

Mundane

Thanks for diving into this! I wasn't able to make it work with [ShouldlyMethods] on the extension method itself, but it works when I add the attribute to the containing class. – Soot

@NielsvanderRest That's very weird if it isn't working on your extension methods. Have you done a clean + rebuild of your solution? What does reflection say? Do you have multiple references to different versions of Shouldly? – Mundane

I'm on the latest version (3.0.2) which seems to only test for the attribute on method.DeclaringType, not the method itself (source). This would explain the behavior I'm seeing. It appears to do the same as the code you posted. I was actually triggered to place the attribute on the class, because that's what I saw in Shouldly's source code. – Soot

@NielsvanderRest Oh wow! Good find! I just looked at my own code and saw that I applied [ShouldlyMethods] to my extension method class, not the methods. I'll update my answer now. – Mundane

Just a note: This approach doesn't work if the assertion method is async. In this case Shouldly won't be able to obtain the should expression. – Gait

@SebastianKrysmanski It works for me when the await is on the same line, at least. – Mundane