How do I test for an error in Haskell?

Question

I want to be able to make sure a function will throw an error when it receives and invalid value. For example, let says I have a function pos that only returns a positive number:

pos :: Int -> Int
pos x
   | x >= 0 = x
   | otherwise = error "Invalid Input"

This is a simplistic example, but I hope you get the idea.

I want to be able to write a test case that will expect an error and consider it a passing test. For example:

tests = [pos 1 == 1, assertError pos (-1), pos 2 == 2, assertError pos (-2)]
runTests = all (== True) tests

[My Solution]

This is what I ended up going with based on @hammar's comment.

instance Eq ErrorCall where
    x == y = (show x) == (show y)

assertException :: (Exception e, Eq e) => e -> IO a -> IO ()
assertException ex action =
    handleJust isWanted (const $ return ()) $ do
        action
        assertFailure $ "Expected exception: " ++ show ex
  where isWanted = guard . (== ex) 

assertError ex f = 
    TestCase $ assertException (ErrorCall ex) $ evaluate f

tests = TestList [ (pos 0) ~?= 0
                 , (pos 1) ~?= 1
                 , assertError "Invalid Input" (pos (-1))
                 ]   

main = runTestTT tests

Heresiarch

error throws an ErrorCall exception. See my answer here for how to test for exceptions using HUnit. – Vaccinate

@Vaccinate Ah, I wasn't sure if that would work in this case. Have you considered submitting that to the HUnit project? It would be nice if it was built in. – Heresiarch

(N.B. all (== True) == all id == and.) – Patrizius

Answer 1

OP's solution defines assertException, but it looks like Test.HUnit.Tools.assertRaises from testpack is also usable here.

I added the msg argument to assertError to match how assertRaises works, and included selective imports so noobs like me can learn where commonly used things are imported from.

import Control.Exception (ErrorCall(ErrorCall), evaluate)
import Test.HUnit.Base  ((~?=), Test(TestCase, TestList))
import Test.HUnit.Text (runTestTT)
import Test.HUnit.Tools (assertRaises)

pos :: Int -> Int
pos x
   | x >= 0 = x
   | otherwise = error "Invalid Input"

instance Eq ErrorCall where
    x == y = (show x) == (show y)

assertError msg ex f = 
    TestCase $ assertRaises msg (ErrorCall ex) $ evaluate f

tests = TestList [
  (pos 0) ~?= 0
  , (pos 1) ~?= 1
  , assertError "Negative argument raises an error" "Invalid Input" (pos (-1))
  ]   

main = runTestTT tests

Freeboot

I love this answer! But do you have an idea on how else to implement this? The HUnit Tools library you're using here hasn't been maintained for years. – Ricoricochet

Answer 2

There are several ways to handle errors in Haskell. Here is an overview: http://www.randomhacks.net/articles/2007/03/10/haskell-8-ways-to-report-errors

[Edit]

The first example shows how to catch errors, e.g.

half :: Int -> Int 
half x = if even x then x `div` 2 else error "odd"

main = do catch (print $ half 23) (\err -> print err)

That said, this kind of error handling is better suited for IO stuff, in pure code like yours Maybe, Either or something similar is usually a better choice. It could be as simple as...

pos :: Int -> Maybe Int
pos x
   | x >= 0 = Just x
   | otherwise = Nothing

tests = [pos 1 == Just 1
        ,pos (-1) == Nothing
        ,pos 2 == Just 2
        ,pos (-2) == Nothing
        ]

main = print $ and tests

... if you don't need an error type.

Homolographic

A useful link, but it doesn't answer the question of how to test errors. – Acaudal

The problem with this answer is that it requires me to change my functions to fit the tests, which is not really the purpose of tests. I want my functions to remain unchanged. – Heresiarch

If your functions are pure, they should avoid to call error, if possible. If they are in IO, catch works fine. If this doesn't help, write a wrapper function transforming errors to Maybe or so. I guess for testing purposes you could even use the infamous unsafePerformIO. – Homolographic