xUnit adding Trait to CollectionDefinition

Question

In xUnit and Visual Studio, I would like to group tests marked with the [Collection("DB")] attribute in the Test Explorer. I can group test by the [Trait("Collection", "DB")] attribute only. Is there any way how to assign a specific Trait to all tests with [Collection("DB")] attribute?

Update: I have added xUnit issue #799.

Mahdi

This is definitely a feature request. Not sure where you would stop though, there are other attributes and you could think of external attributes, or whatever patterns (e.g. naming conventions) you might want to match again. You'd have a combinatorial explosion in the traits view without a UI to pick what you want. In the end it might be more of a feature to request to the VS team for the Test Explorer (i.e. a way to specify rich queries for tests) than a feature for xUnit to half-implement. – Prissie

Answer 1

Copied from http://mac-blog.org.ua/xunit-category-trait/.

using System;
using System.Collections.Generic;
using System.Linq;
using Xunit;
using Xunit.Abstractions;
using Xunit.Sdk;
using XunitCategoriesSample.Traits;

namespace XunitCategoriesSample.Traits
{
    public class CategoryDiscoverer : ITraitDiscoverer
    {
        public const string KEY = "Category";

        public IEnumerable<KeyValuePair<string, string>> GetTraits(IAttributeInfo traitAttribute)
        {
            var ctorArgs = traitAttribute.GetConstructorArguments().ToList();
            yield return new KeyValuePair<string, string>(KEY, ctorArgs[0].ToString());
        }
    }

    //NOTICE: Take a note that you must provide appropriate namespace here
    [TraitDiscoverer("XunitCategoriesSample.Traits.CategoryDiscoverer", "XunitCategoriesSample")]
    [AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
    public class CategoryAttribute : Attribute, ITraitAttribute
    {
        public CategoryAttribute(string category) { }
    }
}

namespace XunitCategoriesSample
{
    public class Class1
    {
        [Fact]
        [Category("Jobsearcher")]
        public void PassingTest()
        {
            Assert.Equal(4, Add(2, 2));
        }

        [Fact]
        [Category("Employer")]
        public void FailingTest()
        {
            Assert.Equal(5, Add(2, 2));
        }

        int Add(int x, int y)
        {
            return x + y;
        }
    }
}

NOTICE you must provide right namespaces in TraitDiscoverer attribute.

But here is more, lets make even more specialized attributes:

public class JobsearcherTraitDiscoverer : ITraitDiscoverer
{
    public const string VALUE = "Jobsearcher";

    public IEnumerable<KeyValuePair<string, string>> GetTraits(IAttributeInfo traitAttribute)
    {
        yield return new KeyValuePair<string, string>(CategoryDiscoverer.KEY, VALUE);
    }
}

[TraitDiscoverer("XunitCategoriesSample.Traits.JobsearcherTraitDiscoverer", "XunitCategoriesSample")]
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public class JobsearcherTraitAttribute : Attribute, ITraitAttribute
{
    public JobsearcherTraitAttribute()
    {
    }
}

So from now on you will be able to just type [JobsearcherTrait]

Links:

https://github.com/xunit/xunit/issues/394 - discussion about why TraitAttribute was marked as sealed

https://github.com/xunit/samples.xunit/tree/master/TraitExtensibility - sample by xunit how to make custom attributes

https://github.com/wespday/CategoryTraits.Xunit2 - one more sample

https://github.com/xunit/xunit/blob/47fdc2669ae6aa28f6d642e202840193dfc7dbd7/test/test.xunit.execution/Common/TraitHelperTests.cs - xunit test sample of implementing custom attributes

Xenocrates

It's simplifies the problem a bit, but does not solve it. I still have to mark the method [Category("DB")] [Collection("DB")]. Note, Collection is a special purpose xUnit attribute. – Mahdi

I think It is possible to do that with code generation: append the extra [Category("DB")] to all methods if the class is attributed [Collection("DB")]. But I don't think it worth it in regular cases. Asking for a feature request is more appropriate. – Xenocrates

Answer 2

In the Xunit.Sdk.ITraitDiscoverer interface GetTraits method's argument 'traitAttribute' is having actual attribute value, but unfortunately the is no direct way to get it as Xunit.Abstractions.IAttributeInfo has no getter which is weird. Here is just another solution without calling GetConstructorArguments()

Enum for exact categories we need

public enum Category
{
    UiSmoke,
    ApiSmoke,
    Regression
}

Custom attribute definition

[TraitDiscoverer("Automation.Base.xUnit.Categories.CategoryDiscoverer", "Automation.Base")]
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = true)]
public class TestCategoryAttribute : Attribute, ITraitAttribute
{
    public string Category { get; }

    public TestCategoryAttribute(Category category)
    {
        Category = category.ToString();
    }
}

And here is the category resolver/discoverer

public sealed class CategoryDiscoverer : ITraitDiscoverer
{
    public const string Key = "Category";

    public IEnumerable<KeyValuePair<string, string>> GetTraits(IAttributeInfo traitAttribute)
    {
        var category = traitAttribute.GetNamedArgument<string>(Key);
        yield return new KeyValuePair<string, string>(Key, category);
    }
}

Here is the catch we need to know exact property name in the TestCategoryAttribute type, in discoverer its defined using Key constant.

Anyway, both GetConstructorArguments() and GetNamedArgument() are based on reflection, while discoverer is executed per each test once being run which is not that super-fast.

Abseil

Answer 3

You can do this:

enum Collection {
  DB,
  File,
  // Others
}

And later at class or method level:

[Trait(nameof(Collection), nameof(Collection.DB))]

Clean and simple

Lemke

Where's the Collection? – Mahdi

Well, if you really need to specify "Collection", you can use "Collection" instead of "TestCategory", as I just did by editing my "answer"... – Lemke

The Collection in the question is the xUnit attribute, see xunit.github.io/docs/shared-context.html#collection-fixture. Not just a trait name. – Mahdi

I'm just giving another way to face/solve the problem. – Lemke

The question is about using the CollectionAttribute to group tests, that's not the purpose of it. Maybe using both attributes together (if a single test context and shared tests are needed) can be the right choice. – Lemke

Yep, using both attributes is what I am doing right now. The question is how to avoid that - how to use just one attribute (either Collection or any new one). – Mahdi