AutoFixture CompositeDataAttribute does not work with PropertyDataAttribute

Question

I'm trying to create AutoPropertyDataAttribute based on CompositeDataAttribute from this example AutoFixture: PropertyData and heterogeneous parameters.

It works with single set of parameters, but fails with more sets of parameters. Here is code:

public static IEnumerable<object[]> NumericSequence
{
    get
    {
        yield return new object[] {1};
        //yield return new object[] {2};
    }
}

[Theory]
[AutoPropertyData("NumericSequence")]
public void Test(int? p1, int? p2, int? p3)
{
    Assert.NotNull(p1);
    Assert.NotNull(p2);
}

public class AutoPropertyDataAttribute : CompositeDataAttribute
{
    public AutoPropertyDataAttribute(string propertyName)
        : base(
              new DataAttribute[] { 
                  new PropertyDataAttribute(propertyName), 
                  new AutoDataAttribute()
              })
    {
    }
}

Trying to uncomment the second yield will break test with message:

System.InvalidOperationException: Expected 2 parameters, got 1 parameters
   at Ploeh.AutoFixture.Xunit.CompositeDataAttribute.<GetData>d__0.MoveNext()
   at Xunit.Extensions.TheoryAttribute.<GetData>d__7.MoveNext()
   at Xunit.Extensions.TheoryAttribute.EnumerateTestCommands(IMethodInfo method)

Same happens with ClassDataAttribute

Target

What is the role of the second yield? It provides the data for p2 or it provides the data for p1 on the 2nd run? – Yankee

@nikos it provides the data for p1 on the 2nd run – Target

Thanks for reporting this. It might be a bug in the CompositeDataAttribute class defined in AutoFixture. However this class is programmed against the base of all data theories, the DataAttribute class, so it looks like a violation of the LSP in PropertyData and ClassDataAttribute classes defined in xUnit.net. I will keep this thread updated. – Yankee

FWIW, there is no LSP violation in xUnit.net PropertyData or ClassData. I have provided an answer below. Hope that helps :) – Yankee

Answer 1

What actually happens

The NumericSequence [PropertyData] defines two iterations.

The composition of NumericSequence [PropertyData] with [AutoData] assumes that there is enough data on each iteration.

However, the actual composition is:

1st iteration:  [PropertyData], [AutoData]

2nd iteration:  [PropertyData], [n/a]

That's why in the 2nd iteration you eventually run out of data.

Composition

The CompositeDataAttribute respects the LSP in a sense that it is programmed against the base of all data theories, the DataAttribute class.

(That is, there is no assumption that all attributes are composed with [AutoData] at the end.)

For that reason, it can't simply jump from the 2nd iteration to the 1st iteration and grab some [AutoData] values – that would break the LSP.

What you could do

Make the actual composition look like:

1st iteration:  [PropertyData], [AutoData]

2nd iteration:  [PropertyData], [AutoData]

By defining two properties:

public static IEnumerable<object[]> FirstPropertyData { get { 
    yield return new object[] { 1 }; } }

public static IEnumerable<object[]> OtherPropertyData { get { 
    yield return new object[] { 9 }; } }

And then, the original test can be written as:

[Theory]
[AutoPropertyData("FirstPropertyData")]
[AutoPropertyData("OtherPropertyData")]
public void Test(int n1, int n2, int n3)
{
}

The test executes twice and n1 is always supplied by [PropertyData] while n2 and n3 are always supplied by [AutoData].

Yankee

I don't understand this explanation. I would think that CompositeAutoData was supposed to use the composed attributes as data sources in order to build what's basically a table (two-dimensional matrix) of values. As long as there's enough data to fill all cells in the table, it should be able to work. Thus, while CompositeAutoData can't expect the last attribute to be an AutoData, it still ought to ask to see if it can provide the requested values. Basically, I'd expect the first composed attribute to determine the number of rows, and the rest to fill in the missing cells. – Underclothing

What I am trying to explain is that on the 2nd iteration the current example runs out of data.. The composite can't just jump from the 2nd iteration to the 1st iteration and supply data.. If that's the case then all the tests in this class should fail. – Yankee

While my answer describes the current behavior, the two-dimensional matrix idea sounds great! But then, we would end up selecting data randomly after some point. – Yankee

This answer applies for xUnit2 and MemberData as well. – Spense

Answer 2

I ran into this issue and decided to implement a custom DataAttribute to solve the problem. I couldn't use either attribute as a base class (reasons below) so I just took the things I needed from the source of each. Thank you OSS :)

Things to note:

• I wanted to change the semantics slightly so that I had the option of yielding single objects rather than arrays. Just makes code look neater for single-object parameters. This meant I couldn't use PropertyDataAttribute as a base class
• The fixture needs to be created every time a new set of parameters is generated. This meant I couldn't use AutoDataAttribute as a base class

Gist

Or inline source below:

public class AutoPropertyDataAttribute : DataAttribute
{
    private readonly string _propertyName;
    private readonly Func<IFixture> _createFixture;

    public AutoPropertyDataAttribute(string propertyName)
        : this(propertyName, () => new Fixture())
    { }

    protected AutoPropertyDataAttribute(string propertyName, Func<IFixture> createFixture)
    {
        _propertyName = propertyName;
        _createFixture = createFixture;
    }

    public Type PropertyHost { get; set; }

    private IEnumerable<object[]> GetAllParameterObjects(MethodInfo methodUnderTest)
    {
        var type = PropertyHost ?? methodUnderTest.DeclaringType;
        var property = type.GetProperty(_propertyName, BindingFlags.Static | BindingFlags.Public | BindingFlags.FlattenHierarchy);

        if (property == null)
            throw new ArgumentException(string.Format("Could not find public static property {0} on {1}", _propertyName, type.FullName));
        var obj = property.GetValue(null, null);
        if (obj == null)
            return null;

        var enumerable = obj as IEnumerable<object[]>;
        if (enumerable != null)
            return enumerable;

        var singleEnumerable = obj as IEnumerable<object>;
        if (singleEnumerable != null)
            return singleEnumerable.Select(x => new[] {x});

        throw new ArgumentException(string.Format("Property {0} on {1} did not return IEnumerable<object[]>", _propertyName, type.FullName));
    }

    private object[] GetObjects(object[] parameterized, ParameterInfo[] parameters, IFixture fixture)
    {
        var result = new object[parameters.Length];

        for (int i = 0; i < parameters.Length; i++)
        {
            if (i < parameterized.Length)
                result[i] = parameterized[i];
            else
                result[i] = CustomizeAndCreate(fixture, parameters[i]);
        }

        return result;
    }

    private object CustomizeAndCreate(IFixture fixture, ParameterInfo p)
    {
        var customizations = p.GetCustomAttributes(typeof (CustomizeAttribute), false)
            .OfType<CustomizeAttribute>()
            .Select(attr => attr.GetCustomization(p));

        foreach (var c in customizations)
        {
            fixture.Customize(c);
        }

        var context = new SpecimenContext(fixture);
        return context.Resolve(p);
    }

    public override IEnumerable<object[]> GetData(MethodInfo methodUnderTest, Type[] parameterTypes)
    {
        foreach (var values in GetAllParameterObjects(methodUnderTest))
        {
            yield return GetObjects(values, methodUnderTest.GetParameters(), _createFixture());
        }
    }
}

Deliciadelicious

This is awesome! Thank you. I made some tweaks so that I could provide multiple class properties and get a combination of parameters (e.g. first property with 3 cases and second property with 2 cases will generate 6 different test cases). This is useful when you already have a paramaterized test method and want to test two different implementations. Also, I combined it with AutoNSubstituteCustomization to get mocking support. – Dunsinane

Answer 3

As a workaround you can restructure the AutoPropertyDataAttribute a bit and use the CompositeDataAttribute internally, rather than deriving from it. Derive from the PropertyDataAttribute instead:

public class AutoPropertyDataAttribute : PropertyDataAttribute
{
    public AutoPropertyDataAttribute(string propertyName)
            : base(propertyName)
    {
    }

Then override the GetData method to loop over the values returned by the PropertyDataAttribute, and leverage AutoFixture's InlineAutoData(which derives from CompositeDataAttribute) to fill in the rest of the parameters:

    public override IEnumerable<object[]> GetData(System.Reflection.MethodInfo methodUnderTest, Type[] parameterTypes)
    {
        foreach (var values in base.GetData(methodUnderTest, parameterTypes))
        {
            // The params returned by the base class are the first m params, 
            // and the rest of the params can be satisfied by AutoFixture using
            // its InlineAutoDataAttribute class.
            var iada = new InlineAutoDataAttribute(values);
            foreach (var parameters in iada.GetData(methodUnderTest, parameterTypes))
                yield return parameters;
        }
    }

The outer loop iterates over the values returned by the PropertyData (each iteration is a row, with some of the cells filled in). The inner loop fills in the remaining cells.

It's not the prettiest thing, but it seems to work. I like Mark's idea to have AutoFixture try to fill in the remaining cells. One less piece of glue code to write :)

Hope this helps,
Jeff.

Fantast

Doesn't work anymore with xunit.net 2.0 since PropertyDataAttribute has been deprecated and its replacement MemberDataAttribute is sealed. This answer worked for me. – Dunsinane