Perl hash of hashes of hashes of hashes... is there an 'easy' way to get an element at the end of the list?

Question

I have a Perl hash of hashes of ... around 11 or 12 elements deep. Please forgive me for not repeating the structure below!

Some of the levels have fixed labels, e.g. 'NAMES', 'AGES' or similar so accessing these levels are fine as I can use the labels directly, but I need to loop over the other variables which results in some very long statements. This is an example of half of one set of loops:

foreach my $person (sort keys %$people) {
        foreach my $name (sort keys %{$people->{$person}{'NAMES'}}) {
            foreach my $age (sort keys %{$people->{$person}{'NAMES'}{$name}{'AGES'}}) {
                . . . # and so on until I get to the push @list,$element; part

This is just an example, but it follows the structure of the one I have. It might be shorter not to have the fixed name sections (elements in caps) but they are required for reference purposes else where.

I tried to cast the elements as hashes to shorten it at each stage, e.g. for the second foreach I tried various forms of:

foreach my $name (sort keys %{$person->{'NAMES'}})

but this didn't work. I'm sure I've seen something similar before, so the semantics may be incorrect.

I've studied pages regarding Hash of Hashes and references to hashes and their elements and so on without luck. I've seen examples of while each loops but they don't seem to be particularly shorter or easier to implement. Maybe there is just a different method of doing this and I'm missing the point. I've written out the full set of foreach loops once and it would be great if I don't have to repeat it another six times or so.

Of course, there may be no 'easy' way, but all help appreciated!

Pellerin

It sounds like your code would benefit from a different model of your data. I'm having a hard time conceiving of a real-world situation corresponding to 11 or 12 levels of hashes. For example, even if a person has multiple names, surely they should not have multiple ages? So AGE should just be another hash key at the person level. Perhaps using Perl's object-oriented capabilities would help: broadcast.oreilly.com/2008/11/…. – Avoirdupois

@WinnieNicklaus, XML often has nodes 12 levels deep. – Milka

This was just an example dataset, it doesn't actually correlate to people but I used that just to show the structure of the code. The data corresponds to parts of a database and refers to a set of ids that are better reduced at an early stage, else will cause the program to slow down later on. It's tediously long, but works for what it represents :) – Pellerin

Why does a name of haves ages (plural)??? – Milka

It's an example... example! T_T – Pellerin

It's making it impossible to come up with proper variable names. Could you give something that makes sense? – Milka

@Milka How about if it were relating to an animal - a census of pets could have the Animal,Breed,Name,Age and so on. – Pellerin

Country->County->City->Street->House->Person(Name,Age) – Gallivant

Interesting how much Mb of data you hold in this tree? – Justicz

There is an easy way. Just traverse it recursively (answer follows). – Heir

@PSIAlt at the moment I have one branch as I'm testing my code, but it is going to contain data pulled from a database that is... big. – Pellerin

Answer 1

$person is the key, to shorten things for the inner loops you need to assign the value to something:

foreach my $person_key (sort keys %$people) {
    my $person = $people->{$person_key};
    my $names  = $person->{NAMES};
    foreach my $name (sort keys %$names) {

Gallivant

ahh this is a good idea! I could do $names = $people->{$person_key}{NAMES}; – Pellerin

Answer 2

Also you can work with each keyword. This definetly should help.

while( my ($person, $val1) = each(%$people) ) {
    while( my ($name, $val2) = each(%$val1) ) {
        while( my ($age, $val3) = each(%$val2) ) {
            print $val3->{Somekey};

Justicz

Answer 3

If you want to build a somewhat more flexible solution, you could traverse the data tree recursively. Consider this example data tree (arbitrary depth):

Example data

my %people = (
    memowe => {
        NAMES => {
            memo        => {AGE => 666},
            we          => {AGE => 667},
        },
    },
    bladepanthera => {
        NAMES => {
            blade       => {AGE => 42},
            panthera    => {AGE => 17},
        },
    },
);

From your question I concluded you just want to work on the leaves (AGEs in this case). So one could write a recursive traverse subroutine that executes a given subref on all leaves it could possibly find in key-sorted depth-first order. This subref gets the leave itself and a path of hash keys for convenience:

Preparations

sub traverse (&$@) {
    my ($do_it, $data, @path) = @_;

    # iterate
    foreach my $key (sort keys %$data) {

        # handle sub-tree
        if (ref($data->{$key}) eq 'HASH') {
            traverse($do_it, $data->{$key}, @path, $key);
            next;
        }

        # handle leave
        $do_it->($data->{$key}, @path, $key);
    }
}

I think it's pretty clear how this guy works from the inlined comments. It would be no big change to execute the coderef on all nodes and not the leaves only, if you wanted. Note that I exceptionally added a prototype here for convenience because it's pretty easy to use traverse with the well-known map or grep syntax:

Executing stuff on your data

traverse { say shift . " (@_)" } \%people;

Also note that it works on hash references and we initialized the @path with an implicit empty list.

Output:

42 (bladepanthera NAMES blade AGE)
17 (bladepanthera NAMES panthera AGE)
666 (memowe NAMES memo AGE)
667 (memowe NAMES we AGE)

The given subroutine (written as a { block }) could do anything with the given data. For example this more readable push subroutine:

my @flattened_people = ();

traverse {
    my ($thing, @path) = @_;
    push @flattened_people, { age => $thing, path => \@path };
} \%people;

Heir

Answer 4

You could use Data::Walk, which is kind of File::Find for data structures.

Walleye