Is it possbile to test for expected errors when the testee exits with failure using TAP within Perl?

Question

Suppose you're running some unit tests and you want to see if the method ( or script or function or whatever) you're testing fails as it should. How do you setup such a test? I'm hoping for something like this:

ok( $obj->method($my, $bad, $params) == DEATH, 'method dies as expected');

although I don't see how it would work since method dies when passed bad parameters and the test script stops.

Is there another way?

Rollet

Answer 1

Have you tried Test::Exception? dies_ok should do what you want. Eg:

# Check that something died - we do not care why
dies_ok { $foo->method } 'expecting to die';

Deary

Answer 2

I recommend Test::Fatal rather than Test::Exception.

Test::Exception has been around for a long time, so a lot of existing test suites use it, but Test::Fatal is easier to learn. Test::Fatal exports only 1 function: exception. This runs the associated code and returns the exception that it threw, or undef if it ran without error. You then test that return value using any of the normal Test::More functions like is, isnt, like, or isa_ok.

Test::Exception requires you to learn its own testing functions like throws_ok and dies_ok, and remember that you don't put a comma between the code and the test name.

So, your example would be:

use Test::More;
use Test::Fatal;

my $obj = ...;

isnt(exception { $obj->method($my, $bad, $params) },
     undef, 'method dies as expected');

Or you could use like to match the expected error message, or isa_ok to test if it threw the correct class of exception object.

Test::Fatal just gives you more flexibility with less learning curve than Test::Exception.

Crumpet

Answer 3

There's really no need to use a module to do this. You can just wrap the call that you expect to fail in an eval block like so:

ok !eval {$obj->method($my, $bad, $params); 1}, 'method dies as expected';

If all goes well in the eval, it will return the last statement executed, which is 1 in this case. if it fails, eval will return undef. This is the opposite of what ok wants, so ! to flip the values.

You can then follow that line with a check of the actual exception message if you want:

like $@, qr/invalid argument/, 'method died with invalid argument';

Growler

The problem with this is that there are edge cases that can be tricky to get right, and the code that handles them correctly is awkward to repeat in your tests. See rjbs's explanation of Test::Fatal. – Crumpet

@Crumpet => There are NO edge cases that could cause the ok line to fail. The only corner case that I can think of here is if there is an eval in a DESTROY that clobbers the error string. And that will only make the like test fail. But that is arguably then catching a bug which should be fixed. None of the examples in Test::Fatal or Text::Exception seem to make this any easier, they just hide !eval{...; 1} behind a function call. Modules are good, but adding a dependency just to wrap such a simple construct has dubious value. – Growler

There's also the possibility that Test::More (or some other testing library) will do an eval and clobber $@ before you look at it. – Crumpet