QuickCheck exit status on failures, and cabal integration

Question

I'm trying to understand how to integrate some quickcheck tests with cabal. This gist suggests that the quickCheck function returns non-zero status on failure, but I am not getting that behavior, so using cabal's exitcode-stdio-1.0 test-suite type doesn't seem to work for me unless I want to call error all over my tests.

The cabal user guide also mentions a detailed-1.0 test-suite, but AFAICT this doesn't exist yet. Is that still the case?

It seems from answers like this one that a lot of people are using the test-framework package. That's overkill for me, but is that what I should use?

I'm left kind of unsatisfied by this situation.

Versions of things I'm using:

cabal-install version 0.10.2
using version 1.10.1.0 of the Cabal library
QuickCheck-2.4.1.1

Combo

Answer 1

Looking at quickCheck's implementation, it indeed never exits the program. However, you can easily implement this behaviour using quickCheckResult:

import Control.Monad
import Test.QuickCheck
import Test.QuickCheck.Test
import System.Exit

main :: IO ()
main = do
  result <- quickCheckResult prop
  unless (isSuccess result) exitFailure

My understanding is that detailed-1.0 is not considered ready for general use yet, and that exitcode-stdio-1.0 is still the recommended testing solution for now.

Blowfly

I know it's been a 3+yrs, but I just spent way too much time trying to figure out that you also need to import Test.QuickCheck.Test for isSuccess to "be in scope". In case anyone is also having that problem. On a side note, is there a way to make it nicer? (ie. not have to do both import Test.QuickCheck but also Test.QuickCheck.Test?) – Linet

@Linet It has apparently been ameliorated at some point: isSuccess is now in Test.QuickCheck. – Cleocleobulus

Answer 2

I used test-framework in the latest version of my Decimal package. It was not overkill; it did just what was wanted. Take a look at the complete source code if you want an example of how to use it.

Heteromorphic

Thanks for the example, I think I'll go that route – Combo