Quickcheck for non-boolean tests

Question

I am using QuickCheck to test my code for some numeric calculations. Basically I have an exact function and several approximations of it that are much more efficient.

I'm currently implementing the properties I want to test something like:

prop_blah input = (abs $ (exact input)-(approx input)) < threshold

But it would be really nice to know exactly how accurate each of the approximation algorithms is and compare them with each other. One easy way to do this would be to get reports of the mean and standard deviation of the left hand side of the inequality. Is this somehow possible?

Heatstroke

Well, you can still use QuickCheck's framework for generating random inputs. For calculating statistics, you might like the obviously named statistics package. – Mckeon

I think you need some proof that approx will not be a factor x away from the correct answer, with that knowledge the quickcheck property is trivial. That is, it might help to analyze your approx function theoretically. – Shaven

@Shaven I agree that such a proof would be nice, but unfortunately these algorithms typically do pretty well, but have poor worst case performance. Sort of like an heuristic for NP-complete problems. – Heatstroke

Answer 1

If you only need it to be printed out, you should check QuickCheck Callbacks which are performed after a single test. Their definition is located in Test.QuickCheck.Property

Otherwise you could use the Function collect :: (Show a, Testable prop) => a -> prop -> Property located in Test.QuickCheck.Property.

let a = (abs $ (exact input)-(approx input))
in collect a (a < threshold)

This way you geld at least a string representation of the approximation and also get to know how many of the single tests give the same approximation.

You could even get rid of the approximation quality and just list the factors by doing:

prop = collect (abs $ (exact input)-(approx input)) True

Tc