Why is an MD5 hash created by Python different from one created using echo and md5sum in the shell?

Question

A Python MD5 hash is different than the one created by the md5sum command on the shell. Why?

>>> import hashlib
>>> h = hashlib.md5()
>>> h.update("mystringforhash")
>>> print h.hexdigest()
86b6423cb6d211734fc7d81bbc5e11d3 # Result from Python


$ echo mystringforhash | md5sum
686687dd68c5de717b34569dbfb8d3c3  - # Result on the shell

Flouncing

Answer 1

echo appends a \n since you usually do not want lines not ending with a linebreak in your shell (it looks really ugly if the prompt does not start at the very left).
Use the -n argument to omit the trailing linebreak and it will print the same checksum as your python script:

> echo -n mystringforhash | md5sum
86b6423cb6d211734fc7d81bbc5e11d3  -

Ennoble

Actually this is one of the big examples I use when I tell people to use more Python or higer level languages instead of shell scripts for work that is typically thought as better done in shell scripts. The nature of mixed data and code, and a different syntax for each command all make shell scripts invisibly error prone – Southwestwardly

If only thing given is "a shell" you cannot trust echo to have a workable -n flag. POSIX says following about echo: "If the first operand is -n, or if any of the operands contain a <backslash> character, the results are implementation-defined." (source: pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html). Use printf instead. – Nel

The problem is not in echo, but in md5sum (now md5 on Mac) and shasum that is adding \n to the end – Wilderness

@Punnerud: No. The output of md5sum doesn't matter here. The input does. And without -n, echo appends a linebreak, which results in a different hash. – Ennoble