Using Perl, how do I decode or create those %-encodings on the web?

Question

I need to handle URI (i.e. percent) encoding and decoding in my Perl script. How do I do that?

---

This is a question from the official perlfaq. We're importing the perlfaq to Stack Overflow.

Sophomore

Answer 1

This is the official FAQ answer minus subsequent edits.

Those % encodings handle reserved characters in URIs, as described in RFC 2396, Section 2. This encoding replaces the reserved character with the hexadecimal representation of the character's number from the US-ASCII table. For instance, a colon, :, becomes %3A.

In CGI scripts, you don't have to worry about decoding URIs if you are using CGI.pm. You shouldn't have to process the URI yourself, either on the way in or the way out.

If you have to encode a string yourself, remember that you should never try to encode an already-composed URI. You need to escape the components separately then put them together. To encode a string, you can use the URI::Escape module. The uri_escape function returns the escaped string:

my $original = "Colon : Hash # Percent %";

my $escaped = uri_escape( $original );

print "$escaped\n"; # 'Colon%20%3A%20Hash%20%23%20Percent%20%25'

To decode the string, use the uri_unescape function:

my $unescaped = uri_unescape( $escaped );

print $unescaped; # back to original

If you wanted to do it yourself, you simply need to replace the reserved characters with their encodings. A global substitution is one way to do it:

# encode
$string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%0x", ord $1 /eg;

#decode
$string =~ s/%([A-Fa-f\d]{2})/chr hex $1/eg;

Sophomore

Mark Stosberg has a relevant blog post: mark.stosberg.com/blog/2010/12/… on the topic. – Kennard

Here's a different use case for those who find themselves on this post while doing bash one-liner where you need to encode in a substitute $ echo "Colon : Hash # Percent %" | perl -MURI::Escape -pe 's/.*/uri_escape($&)/e' Output will look something likes this: Colon%20%3A%20Hash%20%23%20Percent%20%25 – Northeasterly

Answer 2

DIY encode (improving above version):

$string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%02x", ord $1 /eg;

(note the '%02x' rather than only '%0x')

DIY decode (adding '+' -> ' '):

$string =~ s/\+/ /g; $string =~ s/%([A-Fa-f\d]{2})/chr hex $1/eg;

Coders helping coders - bartering knowledge!

Banditry

Answer 3

Maybe this will help deciding which method to choose.

Benchmarks on perl 5.32. Every function returns same result for given $input.

Code:

#!/usr/bin/env perl

my $input = "ala ma 0,5 litra 40%'owej vodki :)";

use Net::Curl::Easy;
my $easy = Net::Curl::Easy->new();
use URI::Encode qw( uri_encode );
use URI::Escape qw( uri_escape );
use Benchmark(cmpthese);

cmpthese(-3, {
    'a' => sub {
        my $string = $input;
        $string =~ s/([^^A-Za-z0-9\-_.!~*'()])/ sprintf "%%%0x", ord $1 /eg;
    },
    'b' => sub {
        my $string = $input;
        $string = $easy->escape( $string );
    },
    'c' => sub {
        my $string = $input;
        $string = uri_encode( $string, {encode_reserved => 1} ); 
    },
    'd' => sub {
        my $string = $input;
        $string = uri_escape( $string );
    },
});

And results:

       Rate      c      d      a      b
c    5618/s     --   -98%   -99%  -100%
d  270517/s  4716%     --   -31%   -80%
a  393480/s  6905%    45%     --   -71%
b 1354747/s 24016%   401%   244%     --

Not surprising. A specialized C solution is the fast. An in-place regex with no sub calls is quite fast, followed closely by a copying regex with a sub call. I didn't look into why uri_encode was so much worse than uri_escape.

Devisable

Answer 4

use URI and it will make URLs that just work.

Mainland

I'm not sure why this was down voted but a one liner example may be needed. $ perl -MURI -E 'my $u=URI->new("https://host:123/"); $u->query_form("foo/slash"=>"1 x", "bar:colon"=>2, foo=>3); say $u;' https://host:123/?foo%2Fslash=1+x&bar%3Acolon=2&foo=3 – Mainland

and the reverse perl -MURI -E 'my $u=URI->new(q{https://host:123/?foo%2Fslash=1+x&bar%3Acolon=2&foo=3}); say join ", ", $u->query_form;' foo/slash, 1 x, bar:colon, 2, foo, 3 – Mainland