Return to old behavior for Perl hash randomization

Question

I am trying to upgrade Perl from version 5.16 to 5.26 and hashes are randomized differently now. Is there a way to return to the old behavior of 5.16? I have an issue with the new behavior, because the existing code and tests depend on the old predictable hash order.

Pent

The old hash behaviour was NOT predictable. Any time the hash grows, there's a chance of getting the current behaviour even in 5.16. Sort the keys if you want a consistent order. – Inarch

Answer 1

Implementation of hashes was seriously reworked in 5.18, see this in perl5180delta.

It's not stated in the (original form of the ) question which part of the extensive overhaul is the problem but it seems that the PERL_PERTURB_KEYS environment variable can help

The PERL_PERTURB_KEYS environment variable allows one to control the level of randomization applied to keys and friends.

When PERL_PERTURB_KEYS is 0, perl will not randomize the key order at all. The chance that keys changes due to an insert will be the same as in previous perls, basically only when the bucket size is changed.
...

A fuller discussion can be found in perlrun, for PERL_HASH_SEED

...
If the option is provided, and PERL_PERTURB_KEYS is NOT set, then a value of '0' implies PERL_PERTURB_KEYS=0/PERL_PERTURB_KEYS=NO and any other value implies PERL_PERTURB_KEYS=2/PERL_PERTURB_KEYS=DETERMINISTIC. See the documentation for PERL_PERTURB_KEYS for important caveats regarding the DETERMINISTIC mode.
...

and in the immediately following PERL_PERTURB_KEYS (my emphasis)

...
(Since Perl 5.18.0) Set to "0" or "NO" then traversing keys will be repeatable from run to run for the same PERL_HASH_SEED. Insertion into a hash will not change the order, except to provide for more space in the hash. When combined with setting PERL_HASH_SEED this mode is as close to pre 5.18 behavior as you can get.
...

So set these environment variables, or set PERL_HASH_SEED and the other one will be set accordingly, for repeated runs of the program to have the same ordering etc. (Note that if keys are added to a hash then it may get randomized in pre-5.18 code as well, in which case setting these variables in 5.18+ won't help. A safer way is to always sort keys.)

I understand the argument for reverting to the old behavior but I have to suggest to read the full documentation and consider security implications.

Also see Algorithmic Complexity Attacks in perlsec.

---

The environment variables for this need be set in the shell, for example in bash as

export PERL_HASH_SEED=0

This can be done in the terminal in which the program will be run, or in .bashrc so you don't have to do it every time when opening a terminal. (But then it affects all scripts run from that shell.) Or, can do it on the same command-line where the program is started

$ PERL_HASH_SEED=0 program.pl arguments

If the progam is started by another program (not directly in a shell), then you can also set it via the %ENV hash in the parent since the environment is inherited by child processes. Then one self-contained way to organize all this is to add a short wrapper script, with

$ENV{PERL_HASH_SEED} = 0;

my @cmd = ...

system( @cmd ) == 0 or ...

Of course there are a number of ways to run a command out of a Perl program other than system, starting with builtin qx (operator form of "backticks") and pipe-open (also see this in perlipc), which allow some control over command's output. Or, better yet, use libraries like IPC::System::Simple, Capture::Tiny, IPC::Run (from simpler to more powerful).

Concerning

Answer 2

The documentation for the change to hash randomization lives here

Below is the setting that will get you a consistent order

Setting PERL_HASH_SEED=0 (exactly one 0) implies PERL_PERTURB_KEYS=0 (hash key randomization disabled)

You can get the keys to return in a consistent order which I suspect isn't enough for your purposes.

Let's test with a small script anyway

# create a hash
my %hash = map { $_ => $_ } 'a' .. 'e';

printHashKeys(\%hash);
printHashKeys(\%hash);
printHashKeys(\%hash);
printHashKeys(\%hash);


sub printHashKeys
{
    my $h = shift ;
    my %hash = %$h;

    for my $key (keys %hash)
    {
        print "$key  " ;
    }
    print "\n";
}

Running with Perl 5.30.2, I get this - keys order is different each time.

$ perl try.pl 
c  a  e  b  d  
a  c  e  b  d  
d  b  e  a  c  
a  c  e  b  d  

Now with PERL_HASH_SEED set to 0. This time they keys come out in a consistent order

$ PERL_HASH_SEED=0 perl try.pl 
b  c  d  e  a  
b  c  d  e  a  
b  c  d  e  a  
b  c  d  e  a  

Pregnable