How can I convert a 48 hex string to bytes using Perl?

Question

I have a hex string (length 48 chars) that I want to convert to raw bytes with the pack function in order to put it in a Win32 vector of bytes.

How I can do this with Perl?

Melone

Answer 1

The steps are:

1. Extract pairs of hexadecimal characters from the string.
2. Convert each pair to a decimal number.
3. Pack the number as a byte.

For example:

use strict;
use warnings;

my $string = 'AA55FF0102040810204080';
my @hex    = ($string =~ /(..)/g);
my @dec    = map { hex($_) } @hex;
my @bytes  = map { pack('C', $_) } @dec;

Or, expressed more compactly:

use strict;
use warnings;

my $string = 'AA55FF0102040810204080';
my @bytes  = map { pack('C', hex($_)) } ($string =~ /(..)/g);

Bromic

Thanks how could i do the reverse operation from @bytes to string – Melone

This does not work correctly. It packs to: aa:20:55:20:ff:20:01:20:02:20:04:20:08:20:10:20:20:20:40:20:80 Why all the 20s in there??? – Enalda

@poosliver: The packed data contains non-printable characters. What you see will depend on how you printed it; I don't know what you did. The value of @bytes as printed by Data::Dump (which uses escape sequences for non-printable characters) is "\xAA", "U", "\xFF", "\1", "\2", "\4", "\b", "\20", " ", "\@", "\x80". – Bromic

Answer 2

my $bytes = pack "H*", $hex;

See perlpacktut for more information.

Caw

I hate that tutorial. Why does it have to be so overly complicated. Why do they give my $s = pack( 'H2' x 10, map { "3$_" } ( 0..9 ) ); as the 'opposite' to my( $hex ) = unpack( 'H*', $mem ); I seriously think they like to convolute things as much as possile. – Incuse

so oneliner example would be printf 414243 | perl -ne 'print pack "H*", $_' to return ABC – Burnie

@Matija: That was the comment where I first really understood the H of pack! I always thought the 2nd param of pack is a number (list). For all others: Considering that pack('H*', 0x065223) and pack('H*', 414243) both produce ABC, we see 2 things: pack('H*', x) does not need a list of nybbles but can even interpret a string of nibbles. And: perl always "thinks" decimal in its number representation. – Shaun

Answer 3

I have the string:

"61 62 63 64 65 67 69 69 6a"

which I want to interpret as hex values, and display those as ASCII chars (those values should reproduce the character string "abcdefghij").

Typically, I try to write something quick like this:

$ echo "61 62 63 64 65 67 69 69 6a" | perl -ne 'print "$_"; print pack("H2 "x10, $_)."\n";'
61 62 63 64 65 67 69 69 6a
a

... and then I wonder, why do I get only one character back :)

First, let me note down that the string I have, can also be represented as the hex values of bytes that it takes up in memory:

$ echo -n "61 62 63 64 65 67 68 69 6a" | hexdump -C
00000000  36 31 20 36 32 20 36 33  20 36 34 20 36 35 20 36  |61 62 63 64 65 6|
00000010  37 20 36 38 20 36 39 20  36 61                    |7 68 69 6a|
0000001a

_(^{NB: Essentially, I want to "convert" the above byte values in memory as input, to these below ones, if viewed by hexdump:}

$ echo -n "abcdefghij" | hexdump -C
00000000  61 62 63 64 65 66 67 68  69 6a                    |abcdefghij|
0000000a

... which is how the original values for the input hex string were obtained. )_

Well, this Pack/Unpack Tutorial (AKA How the System Stores Data) turns out is the most helpful for me, as it mentions:

The pack function accepts a template string and a list of values [...]

$rec = pack( "l i Z32 s2", time, $emp_id, $item, $quan, $urgent);

It returns a scalar containing the list of values stored according to the formats specified in the template [...]

$rec would contain the following (first line in decimal, second in hex, third as characters where applicable). Pipe characters indicate field boundaries.

    Offset   Contents (increasing addresses left to right)
         0   160  44  19  62| 41  82   3   0| 98 111 120 101 115  32 111 102
              A0  2C  13  3E| 29  52  03  00| 62  6f  78  65  73  20  6f  66
                                            |  b   o   x   e   s       o   f

That is, in my case, $_ is a single string variable -- whereas pack expects as input a list of several such 'single' variables (in addition to a formatting template string); and outputs again a 'single' variable (which could, however, be a sizeable chunk of memory!). In my case, if the output 'single' variable contains the ASCII code in each byte in memory, then I'm all set (I could simply print the output variable directly, then).

Thus, in order to get a list of variables from the $_ string, I can simply split it at the space sign - however, note:

$ echo "61 62 63 64 65 67 68 69 6a" | perl -ne 'print "$_"; print pack("H2", split(/ /, $_))."\n";'
61 62 63 64 65 67 68 69 6a
a

... that amount of elements to be packed must be specified (otherwise again we get only one character back); then, either of these alternatives work:

$ echo "61 62 63 64 65 67 68 69 6a" | perl -ne 'print "$_"; print pack("H2"x10, split(/ /, $_))."\n";'
61 62 63 64 65 67 68 69 6a
abcdeghij

$ echo "61 62 63 64 65 67 68 69 6a" | perl -ne 'print "$_"; print pack("(H2)*", split(/ /, $_))."\n";'
61 62 63 64 65 67 68 69 6a
abcdeghij

Nebulose