Does AutoFixture have a mechanism to check that methods that return any kind of list never return null?

Question

I want to write a test that checks classes within a given namespace. All methods of that classes that return any kind of list have to be checked, if they would return null. If so the test has to fail.

The classes/methods itself have also dependencies (constructor arguments and method parameters) which should be automocked.

Has AutoFixture a mechanism to check that methods that return any kind of list never return null?

Example Class:

public class UserService
{
    private readonly IRemotingFacade _remotingFacade;

    public UserService(IRemotingFacade remotingFacade)
    {
        _remotingFacade = remotingFacade;
    }

    // directly return a list
    public IEnumerable<User> GetUsers()
    {

    }

    // directly return a list, pass method parameters
    public IEnumerable<User> GetUsers(string filter)
    {

    }

    // wrapped list
    public IBusinessResponse<IEnumerable<User>> GetUsers()
    {

    }


    // wrapped list, pass method parameters
    public IBusinessResponse<IEnumerable<User>> GetUsers(string filter)
    {

    }
}

So please conider that the list can be wrapped within another object.

Update

AutoFixture.Idioms covers a lot of this sort of thing. It's not documented yet but as usual for any @ploeh stuff, it's covered in excellent tests, so have a look at the Scenarios. I'm guessing it doesnt do exactly this yet, but it def does lots of things almost the same. – Perot

Answer 1

AutoFixture 3.18.0 introduces a new glue library called Idioms.FsCheck which uses FsCheck to implement a reusable assertion named ReturnValueMustNotBeNullAssertion.

This new assertion verifies (or at least, makes probable) that a method that returns a value (Query) doesn't return null.

Installation

Idioms.FsCheck is available on NuGet:

PM> Install-Package AutoFixture.Idioms.FsCheck

Scenarios

The UserService uses an injected instance of an IRemotingFacade and exposes two^[1] Queries:

• User[] GetUsers()
• User[] GetUsers(int)

Scenario #1: The injected instance of an IRemotingFacade returns null:

[Theory, UnitTestConventions]
public void Scenario1(
    ISpecimenBuilder builder,
    [Frozen]Mock<IRemotingFacade> stub)
{
    stub.Setup(x => x.GetUsers()).Returns((User[])null);

    var sut = from x in new Methods<UserService>() select x.GetUsers();

    var assertion = new ReturnValueMustNotBeNullAssertion(builder);
    Assert.Throws<ReturnValueMustNotBeNullException>(() =>
        assertion.Verify(sut));
}

Scenario #2: The injected instance of an IRemotingFacade doesn't return null:

[Theory, UnitTestConventions]
public void Scenario2(
    ISpecimenBuilder builder,
    [Frozen]Mock<IRemotingFacade> stub,
    User[] users)
{
    stub.Setup(x => x.GetUsers()).Returns(users);

    var sut = from x in new Methods<UserService>() select x.GetUsers();

    var assertion = new ReturnValueMustNotBeNullAssertion(builder);
    Assert.DoesNotThrow(() => assertion.Verify(sut));
}

Scenario 3: If i is -1 GetUsers(int) returns null:

[Theory, UnitTestConventions]
public void Scenario3(
    ISpecimenBuilder builder,
    [Frozen]Mock<IRemotingFacade> stub,
    User[] users)
{
    stub.Setup(x => x.GetUsers()).Returns(users);

    var sut = from x in new Methods<UserService>() 
              select x.GetUsers(default(int));

    var assertion = new ReturnValueMustNotBeNullAssertion(builder);
    Assert.Throws<ReturnValueMustNotBeNullException>(
        () => assertion.Verify(sut));
}

Remarks

If you only have F# 3.1 installed, you may also add an assembly binding redirect in your app.config file:

<dependentAssembly>
  <assemblyIdentity name="FSharp.Core" 
                    publicKeyToken="b03f5f7f11d50a3a" 
                    culture="neutral" />
  <bindingRedirect oldVersion="0.0.0.0-4.3.1.0" 
                   newVersion="4.3.1.0" />
</dependentAssembly>

The UnitTestConventionsAttribute is defined as:

internal class UnitTestConventionsAttribute : AutoDataAttribute
{
    internal UnitTestConventionsAttribute()
        : base(new Fixture().Customize(new AutoMoqCustomization()))
    {
    }
}

The reflection queries are performed with Albedo.

---

^[1] For the demo, I simplified the original code as below:

public class User
{
}

public interface IRemotingFacade
{
    User[] GetUsers();
}

public class UserService
{
    private readonly IRemotingFacade remotingFacade;

    public UserService(IRemotingFacade remotingFacade)
    {
        if (remotingFacade == null)
            throw new ArgumentNullException("remotingFacade");

        this.remotingFacade = remotingFacade;
    }

    public User[] GetUsers()
    {
        return this.remotingFacade.GetUsers();
    }

    public User[] GetUsers(int i)
    {
        if (i == -1)
            return null;

        return this.remotingFacade.GetUsers();
    }
}

Frigging

For all that want to try out this code, you need the following packages: Albedo, AutoFixture, AutoFixture.AutoMoq, AutoFixture.Idioms, AutoFixture.Idioms.FsCheck, AutoFixture.Xunit, FsCheck, Moq, Moq.AutoMock, xunit, xunit.extensions – Update

If you reference the higher-level ones, the rest of the dependencies will get added automatically by NuGet. Although I am writing this out of my head, these should be AutoFixture.Idioms.FsCheck, AutoFixture.Xunit, and AutoFixture.AutoMoq. – Frigging

Answer 2

Ruben Bartelink's comment above is correct. Surprisingly, AutoFixture.Idioms doesn't (yet) have that particular test, although the first idiomatic test introduced to that library was its equivalent on the Command-side: GuardClauseAssertion

However, I think it's an excellent idea (and I don't know why I hadn't thought of that before), so I've now added a new task to the backlog.

Ogletree

I have updated my question. I just wanted to provide some use cases. – Update