Conditional QuickCheck properties

Question

I wrote a QuickCheck property for a function that merges two sorted inputs into a sorted output:

prop_merge xs ys =
    if (sorted xs && sorted ys) then (sorted (merge xs ys)) else True

That is, when the inputs are sorted, the output is sorted as well. It can also be written as:

prop_merge xs ys = not(sorted xs && sorted ys) || (sorted (merge xs ys))

But I don't really like either version. Is there a nicer syntax for "conditional properties" in QuickCheck?

Stewpan

Answer 1

You can use the ==> operator to attach a boolean condition to your properties:

prop_merge xs ys = (sorted xs && sorted ys) ==> sorted (merge xs ys)

This is not only a nicer syntax, but allows QuickCheck to differentiate between test cases where the test was successful and test cases which did not satisfy the precondition. In the latter case, the test is not counted and QuickCheck generates new inputs.

However, in cases where most inputs don't satisfy the condition, this will cause your tests to either run more slowly, or, if enough inputs are discarded, QuickCheck will eventually give up. Since a random list is unlikely to be sorted, this is very likely to happen with the example above:

> quickCheck (prop_merge :: [Int] -> [Int] -> Property)
*** Gave up! Passed only 15 tests.

(Note that with standard boolean operators instead of using ==>, QuickCheck will boost about all tests being passed, when most of them were useless due to a failed precondition)

For this reason, it's generally much better to directly generate only the test cases you need. For simple cases, the Test.QuickCheck.Modifiers module contains several useful newtypes which modify the way inputs are generated. For example, the OrderedList modifier will generate only sorted lists, so we can write your property simply as:

prop_merge (Ordered xs) (Ordered ys) = sorted (merge xs ys)

Craving

I think it would be useful to note explicitly that ==> is not only a nicer syntax, but allows QuickCheck to differentiate between test cases where the test was successful and test cases which did not satisfy the precondition. In the latter case, the test is not counted and QuickCheck generates new inputs. This is what enables QuickCheck to give you the "*** Gave up! Passed only 15 tests." message." If the precondition is expressed via the standard boolean operators, QuickCheck will boost about all tests being passed, when most of them were useless due to a failed precondition. – Superjacent

@cebewee: Thanks for your suggestions. I've edited them in manually since they were rejected by the reviewers. – Craving