How to create a generator with a fixed list of items for FsCheck

Question

I originally tried to create a generator that have the first 5 elements fixed (and on any test using Prop.forAll the first five would always run), but failed in doing so.

Now I am trying to simplify this by having one generator for random data within a range, and one generator for non-random data, i.e., a fixed sequence. It is similar to Gen.constant, except that instead of one value, it is a sequence of values.

I have this (simplified reproducible example, works with NUnit and xUnit):

[<Property(Verbose = true, MaxTest=5)>]
static member MultiplyIdentityCornerCases () =
    Gen.elements [0L; -1L; 1L; Int64.MinValue; Int64.MaxValue]
    |> Arb.fromGen 
    |> Prop.forAll <| fun x -> x = x * 1L

The output is (no idea where the null comes from):

0:
<null>
9223372036854775807L
1:
<null>
-9223372036854775807L
2:
<null>
-9223372036854775807L
3:
<null>
1L
4:
<null>
-9223372036854775807L
Ok, passed 5 tests.

I'd like the output to contain all the five tests in the sequence, preferably, but not necessarily, in order. I know I can do this with NUnit (or any unit testing system) using a testdata provider, but I wonder whether I can do it with FsCheck (or whether I should, perhaps this is a bad idea).

I think using FsCheck is useful, as for the situation where there's more than one function argument, I want it to exhaustively test all combinations of the corner cases arguments I give it. This is hopefully easier with FsCheck than with a testdata provider.

Gnarly

Thinking out loud, one way of fixing this is perhaps using the non-randomness of the number generator (or check how that is done), for integers it seems to work exactly the way I need it to work (see FsCheck output of my earlier question) – Gnarly

Answer 1

I'm not aware that that's possible, but you can do this:

open System
open FsCheck
open FsCheck.Xunit

[<Property>]
let MultiplyIdentityCornerCases () =
    Gen.oneof [
        Gen.elements [Int64.MinValue; -1L; 0L; 1L; Int64.MaxValue]
        Arb.generate ]
    |> Arb.fromGen
    |> Prop.forAll <| fun x -> x = x * 1L

Two generators are passed to Gen.oneof, so each of these will generate approximately half of the values.

Gen.elements ought to pick uniformly from all values in the provided sequence, so it'll use e.g. 0L 20% of the time, but only for those half when Gen.oneof uses Gen.elements.

In other words, each of those 'special' values will be generated 50% * 20% = 10% of the time.

By default, a property runs 100 test cases, so on average, it should generate 10 0L values, 10 Int64.MinValue values, and so on. That should often be good enough.

---

If it isn't, you can always do something like this:

open System
open Xunit
open FsCheck
open FsCheck.Xunit
open Swensen.Unquote

[<Theory>]
[<InlineData(Int64.MinValue)>]
[<InlineData(-1L)>]
[<InlineData( 0L)>]
[<InlineData( 1L)>]
[<InlineData(Int64.MaxValue)>]
let MultiplyIdentityCornerCases x = x =! x * 1L

[<Property>]
let MultiplyIdentityCornerCasesProperty x =
    MultiplyIdentityCornerCases x

Here, you define a Parameterized Test using xUnit.net's [<Theory>] feature, and feed it the five corner cases you are concerned with. When you run the tests, the test runner will run those five test cases.

Furthermore, it'll run MultiplyIdentityCornerCasesProperty because it's annotated with [<Property>], and that function simply calls the other function.

Thorson

I like the second code-snippet best. It's actually pretty neat. – Ferbam

Thanks again, Mark. I like the second approach, provided I could replace the static data for a data provider, but that shouldn't be a problem. For now I've used a variant of your first approach above, which seems to work fine in practice (and making sure that enough tests are run to hit all the probabilities with a certain certainty). – Gnarly