FsCheck integration with NUnit in C#

Question

TL;DR: I'm not able to successfully use FsCheck with NUnit in C#: either:

• it tells me on stdout that the test failed but the test still appears green
• it tells me that it doesn't find any test to run
• or I don't understand how to apply in C# the doc I read

I think a dummy but complete example would help...

---

(More details)

1st step: test remains green

I installed the Nuget package FsCheck.NUnit (2.10.4), and naively tried:

[NUnit.Framework.Test]
public void SomeTest()
{
    // Note the subtle bug: I Reverse only once, because I want the test to fail
    Func<int[],bool> revRevIsOrig = xs => xs.Reverse().SequenceEqual( xs );
    Prop.ForAll(revRevIsOrig).QuickCheck();
}

When I run it as I would run any NUnit test, it ends up green, even though I see on stdout that

Falsifiable, after 3 tests (1 shrink) (StdGen (2129798881,296376481)):
Original:
[|-1; 0|]
Shrunk:
[|1; 0|]

2nd step: test inconclusive

So I went on, found some doc, and noticed that I should use Property instead of Test. So I changed my code to

[FsCheck.NUnit.Property] // <-- the line that changed
public void SomeTest()
{
    Func<int[],bool> revRevIsOrig = xs => xs.Reverse().SequenceEqual( xs );
    Prop.ForAll(revRevIsOrig).QuickCheck();
}

I launched the test from Visual, and it ended up with the status inconclusive. And some log was telling me:

Cannot run tests: No suitable tests found in 'xxx.exe'. Either assembly contains no tests or proper test driver has not been found

3rd step: I notice I didn't understood the doc

When I re-read the doc, I notice that it says that my test method can take argument and should return a property. I'm obviously not doing it since I return nothing.

The sad thing is that I don't understand what I'm actually supposed to do (and I'm not familiar enough with F# to understand the example below...)... (I've blindly tried some random stuff that looked they could make sense, but I never ended up with a red test)

I would really appreciate any pointer to help me get this test red!

Conti

Try Prop.ForAll(revRevIsOrig).QuickCheckThrowOnFailure(); – Laspisa

Answer 1

Try using the QuickCheckThrowOnFailure function

the QuickThrowOnFailure ensures that if the test fails, an exception with the necessary information is raised so the runner know the test failed.

[Test]
public void SomeTest() {
    // Note the subtle bug: I Reverse only once, because I want the test to fail
    Func<int[],bool> revRevIsOrig = xs => xs.Reverse().SequenceEqual( xs );
    Prop.ForAll(revRevIsOrig).QuickCheckThrowOnFailure();
}

Laspisa

Awesome: it works completely as I expected! Thanks a lot! – Conti

Answer 2

The other way to use FsCheck with NUnit (from FsCheck github examples) is

[FsCheck.NUnit.Property(Verbose = true)]
public void SomeTest()
{
    Func<int[],bool> revRevIsOrig = xs => xs.Reverse().SequenceEqual( xs );
    return Prop.ForAll(revRevIsOrig);
}

But this way doesn't report you why your test fails. So the Nkosi's answer now is better but I think it will be fixed somedays.

Plummer

"from FsCheck github examples" Please could you share a link? – Titos

github.com/fscheck/FsCheck/tree/master/examples/… if still relevant – Plummer

Answer 3

Besides other mentioned options you can turn this test to a Property Based Test by using the Property attribute and correct return type (I've included namespaces for clarity).

Note the .ToProperty call to turn it into a Property.

[FsCheck.NUnit.Property]
public FsCheck.Property RevRev_IsOriginal(int[] xs)
{
    Func<bool> revRevIsOrig = () => xs.Reverse().Reverse().SequenceEqual(xs);
    return revRevIsOrig.ToProperty();
}

The framework will generate testdata via the int[] xs method parameter.

This requires nuget packages fscheck and fscheck.nunit (besides NUnit3TestAdapter and Microsoft.NET.Test.Sdk in my .NET Core 3.1 testproject).

Storebought