Test::More is_deeply does not pretty-print array/hashrefs when comparing against strings

Question

When Test::More compares like arrayrefs and hashrefs with each other, the corresponding diagnostic message is really informative and shows the first index where the structures differ, regardless of nesting depth. However, when it is comparing an arrayref or hashref to a simple scalar, it produces a stringified scalar (with a memory address and reference type) in the diagnostic message, which is harder to interpret.

Is there a way to configure Test::More to pretty-print array or hashrefs in a custom way (such as using Data::Dumper)?

Here's an example with two test cases. The first gives you some insight into what is present in your program but unexpected. The second informs the user of a type mismatch between the string and the arrayref, but does not print any items in the arrayref.

#!/usr/bin/env perl
use strict;
use warnings;
use Test::More tests => 2;

is_deeply(
    {
        a => [5],
    },
    {
        a => [5, 6, 8],
    },
    'compare two hashrefs structurally (very informative output)',
);

is_deeply(
    [5, 6, 8],
    "",
    'compare two "incompatible" values structurally (uninformative output)',
);

And the TAP output:

1..2
not ok 1 - compare two hashrefs structurally (very informative output)
#   Failed test 'compare two hashrefs structurally (very informative output)'
#   at test-more-failure.pl line 6.
#     Structures begin differing at:
#          $got->{a}[1] = Does not exist
#     $expected->{a}[1] = '6'
not ok 2 - compare two "incompatible" values structurally (uninformative output)
#   Failed test 'compare two "incompatible" values structurally (uninformative output)'
#   at test-more-failure.pl line 16.
#     Structures begin differing at:
#          $got = ARRAY(0x7fe66b82cde8)
#     $expected = ''
# Looks like you failed 2 tests of 2.

Looking at the implementation of is_deeply in Test::More, there doesn't seem to be way to use a custom pretty-printer or configure the verbosity of the module. At least none that's obvious to me.

Here is what happens when we compare a reference and non-reference:

https://metacpan.org/source/EXODIST/Test-Simple-1.302062/lib/Test/More.pm#L1121

It seems to be calling _format_stack({vals => [...]}) instead of _format_stack(...)

https://metacpan.org/source/EXODIST/Test-Simple-1.302062/lib/Test/More.pm#L1139

Lyman

Answer 1

tl;dr Use is_deeply($this, $that) || diag explain $this on a case by case basis.

Hi. I'm the one to blame for is_deeply. It's deliberately designed to not vomit out a potentially enormous data structure when something fails. Instead it stops at the first difference. For this reason you probably don't want to globally make is_deeply dump its arguments. If the types are wrong, if you expected apples and got zebras, there's not much point in knowing how many zebras and their life stories.

There is no supported way to change its diagnostics, sorry, and it's unlikely there will be. Test::More is being replaced by Test2. Test::More is already implemented on top of Test2, but doesn't take advantage of its features for backwards compatibility reasons.

You can use Test2::Bundle::More for more direct access to Test2's features, but it is not 100% compatible, and it displays similar to how is_deeply does. However, it is much more flexible and you can probably figure out a way to alter its diagnostic behavior. Look into Test2::Compare.

---

Back to your problem... use explain on a case by case basis. explain uses Data::Dumper, configured properly, to dump the data structure. Since Test::More functions return whether they passed or failed, you can write is_deeply($this, $that) || diag explain $this. For example...

my $stuff = [5, 6, 8];
is_deeply $stuff, "" || diag explain $stuff;

not ok 2
#   Failed test at /Users/schwern/tmp/test.plx line 17.
#     Structures begin differing at:
#          $got = ARRAY(0x7f80b3803078)
#     $expected = ''
# [
#   5,
#   6,
#   8
# ]

diag is how you print failure diagnostics (it's a more polite way to print to STDERR).

Theomachy

Answer 2

Try eq_or_diff( $got, $expected, $message ) from Test::Differences, it will print a beautiful representation of your data structures and clearly highlight the exact similarities and differences.

Nagual

If you want a side-by-side diff of the data structures, this is really nice. If you're in development, though, and the data structure has changed such you just want to replace the definition in the test module, diag explain $got provides an easier output to copy and paste. – Dukes