How to get same crypt(3) function in Mac OS X as Linux gcc/gnu crypt(3)? Linux gcc crypt(3) has MD5 and SHA512. Apple Gcc crypt(3) *only* uses DES

Question

I am migrating some c code from Linux to Mac OSX (yosemite). The Mac OSX crypt() function (which is in unistd.h, as I have determined), is not the same as on gcc/gnu in Linux. I have test programs on Linux and Mac OSX, and the crypt() c-library function displays a 34-character string, if you display result using printf. The same code on the Mac OSX, displays only a 13 char string. A bit of research suggests that the difference is apparently the Linux crypt() library routine generates the hash from longer vectors on the gnu/gcc Linux side, using perhaps a different encryption algorithm. Some information also suggests that the Apple Mac OSX c-library crypt() function is only using DES to encrypt the original string plus the salt. I want to have my test code produce the same results, on both Linux and Mac OSX platforms.

Is there a proper crypt() function for Apple Mac OSX, isometric to the gnu/gcc Linux version, perhaps in some open-source encryption library?

Or is there a gnu/gcc equivalent crypt(3) function available for Apple Mac OSX somewhere in the Mac OSX development tools? (I am very new to Mac OSX). I am using the clang compiler, part of the Xcode stuff downloaded from Apple, current for Yosemite 10.10.5, and I am guessing I am not the first person to encounter this anomaly. Thanx for any info.

Ah. just a little edit: The Apple MacOSX stuff is using DES algo, hence the 13-char result, when inspecting the hash. Gnu/gcc uses MD5 algo, hence the 34-char resulting hash. This is explained here: http://www.gnu.org/savannah-checkouts/gnu/libc/manual/html_node/crypt.html#crypt My refined question then; is there a crypt(3) equivalent function for Mac OSX that uses MD5 (or SHA512) instead of DES?

(*** 2nd edit Note: This is getting interesting. DES is poor, but MD5 can be cracked in Kali Linux, using "hashcat"? Recommendation is to go to SHA512 apparently. Details on academic test/verify re. MD5 cracking are here: https://uwnthesis.wordpress.com/2013/08/07/kali-how-to-crack-passwords-using-hashcat/ Still, my question remains. Is there MD5 crypt(3) function for Mac OSX somewhere?? Thx.)

(Pls excuse my rank ignorance of protocol. Mac OS X LLVM/gcc based crypt() fuction is borked junk, hardwired to only use DES, a proven breakable hash, worse than MD5. (Call it with the salt string as $6$, and you will get back a 13 char DES hash. Unbelievable!) I have discovered numerous methods to create password hashes properly (ie. MD5 and SHA512) on Linux platforms (perl, python, etc.) They typically use either the "crypt" lib (same one you get using "-lcrypt" option in gcc on Linux), or "passlib" for python. But my MacBook, just updated to Yosemite 10.10.5 does not even have "passlib"! (My older Fedora box runs Python 2.5.1, current CentOS box runs Python 2.6.6. The nasty little Macbook indicates Python 2.7.10, using "python --version" command. I found this excellent post over at "ServerFault" site: https://serverfault.com/questions/330069/how-to-create-an-sha-512-hashed-password-for-shadow?newreg=df160746fbcd47878d600cbe76ec8b7f

The first python and the perl scripts work on Linux, since they are using glibc crypt(), I am guessing, but no "passlib.hash" seems to exist anywhere, Linux or Mac OS X.

How the heck can I create a decent password hash on this MacBook thing? I am a Mac OS X noob, and since I have confirmed that Apple is using SHA512 password hashes inside its .plist files, I am certain this fuctionality has to exist somewhere on this weird (but lovely) piece of alien hardware. In case one is curious, you can enter this to see your "ShadowHashData" on Yosemite, from the cmd line in terminal: (sorry, forgot reference for this, found it searching last nite, really helpful)

sudo defaults read /var/db/dslocal/nodes/Default/users/<yourusername>.plist ShadowHashData | tr -dc 0-9a-f | xxd -r -p | plutil -convert xml1 - -o -

So it looks like Darwin/Yosemite is using ok encryption. I read some Apple documentation on common-crypto stuff, but there is scant info on how to tweak your gcc to actually point to the library containing this critical stuff. When I determine how to this, I will post the answer here.

Midkiff

You could just use the glibc crypt code directly - you'll need to provide any modified code under the LGPL, but it's all free. – Dantzler

md5sum should be the same on both OSX and Linux – Knight

Does OSX still ship with OpenSSL (albeit deprecated)? If so, openssl passwd -1 generates an md5-compatible crypt hash. – Girl

I have some code which creates MD5 hashes from a string, but it is not the same as what crypt(3) does. Perhaps just install gcc on Mac OS X? I can't be the first person to encounter this. Is there an obvious workaround? – Midkiff

Can't you can use CommonCrypto's CC_MD5? – Majordomo

Linux/gcc crypt() makes use of MD5 hash algo, but it does much more. (From the wikipedia entry for MD5-based crypt()): "First the passphrase and salt are hashed together, yielding an MD5 message digest. Then a new digest is constructed, hashing together the passphrase, the salt, and the first digest, all in a rather complex form. Then this digest is passed through a thousand iterations of a function which rehashes it together with the passphrase and salt in a manner that varies between rounds. The output of the last of these rounds is the resulting passphrase hash." – Midkiff

The developer.apple.com site in its porting notes admits the difference in its Xcode/gcc version of crypt(). Apple states: "The crypt function performs password encryption, based on the NBS Data Encryption Standard (DES). Additional code has been added to deter key search attempts.crypt. OS X's version of the function crypt behaves very similarly to the Linux version, except that it encrypts a 64-bit constant rather than a 128-bit constant. This function is located in the unistd.h header file rather than lcrypt.h." Test prgm on MacBook confirms this very short 13 char passphrase hash. – Midkiff

Welcome to the world of cross-platform development. crypt, as defined by POSIX, doesn't mandate a specific encryption algorithm: The algorithm is implementation-defined. – Thant

Answer 1

OS X doesn't use its crypt function for much of anything. It has it for compatibility with POSIX, which doesn't define how it works, and different platforms have built different solutions over the years. Linux's isn't particularly "correct," it's just another vendor-specific solution. Apple explains their rationale for crypt in crypt(1):

This library (FreeSec 1.0) was developed outside the United States of America as an unencumbered replacement for the U.S.-only libcrypt encryp- tion library. Programs linked against the crypt() interface may be exported from the U.S.A. only if they use crypt() solely for authentication purposes and avoid use of the other programmer interfaces listed above. Special care has been taken in the library so that programs which only use the crypt() interface do not pull in the other components.

If you need cross-platform password hashing, then you need to implement it with a known algorithm, which crypt doesn't provide. In general that means you're going to need to generate your own format, because there really isn't a standard one. I recommend using PBKDF2 rather than just salted SHA2, since it includes stretching as well as salting. Cracking weak SHA512 passwords with John the Ripper is still very fast and easy. With good stretching, even fairly weak passwords can be protected. (From the description of Linux's crypt they seem to be doing something similar to PKBDF2; possibly exactly PBKDF2 without naming it.) Similar algorithms include scrypt and bcrypt. I like PBKDF2 because it's so ubiquitous and recommended by NIST (even though there are very reasonable arguments that bcrypt and scrypt are stronger).

The pieces of PKBDF2 aren't all that complicated, and there are some good implementations of the pieces you need in C with permissive licenses. See MGCryptor for one example that has all the pieces you would require in simple ANSI C with an MIT license. Look particularly at pkcs5_pbkdf2() which may be exactly what you want.

PBKDF2 is widely available on many platforms and languages. OS X provides it through Common Crypto.

You of course could implement your own version of Linux's crypt using Common Crypto as well. But you'll have to be very careful not to copy any (L)GPL code in the process unless you plan to use a compatible license. Personally, I would build a PBKDF2-based solution.

Imco