Unable to decrypt stored encrypted data

Question

I'm running into issues with data I'm trying to store encrypted in my Rails 4 app. I've been looking at lots of questions related to this and there are many hints, feels like I'm almost there, but somehow it just won't decrypt the data. These are the two methods involved:

def encrypt( val, pwd_name )
    cipher = OpenSSL::Cipher.new 'AES-128-CBC'
    cipher.encrypt
    iv = cipher.random_iv

    pwd = encryptor_pwds[ pwd_name ]
    salt = OpenSSL::Random.random_bytes 16
    iter = 20000
    key_len = cipher.key_len
    digest = OpenSSL::Digest::SHA256.new

    key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest)
    cipher.key = key

    encrypted = cipher.update val
    encrypted << cipher.final

    encrypted = Base64.encode64( encrypted ).encode('utf-8')
    iv = Base64.encode64( iv ).encode('utf-8')
    salt = Base64.encode64( salt ).encode('utf-8')

    return { str: encrypted, iv: iv, salt: salt }
end



def decrypt( str, iv, salt, pwd_name )
    cipher = OpenSSL::Cipher.new 'AES-128-CBC'
    cipher.decrypt

    str = Base64.decode64( str )
    iv = Base64.decode64( iv )
    salt = Base64.decode64( salt )

    cipher.iv = iv

    pwd = encryptor_pwds[ pwd_name ]
    salt = salt
    iter = 20000
    key_len = cipher.key_len
    digest = OpenSSL::Digest::SHA256.new

    key = OpenSSL::PKCS5.pbkdf2_hmac(pwd, salt, iter, key_len, digest)
    cipher.key = key

    decrypted = cipher.update str
    decrypted << cipher.final
    return decrypted
end

And the I modified the read/writes to for example this:

def email=(email)
    unless email.nil?
        set = encrypt(email, :email)
        write_attribute( :email, set[:str] )
        write_attribute( :email_iv, set[:iv] )
        write_attribute( :email_salt, set[:salt] )
    else
        write_attribute( :email, nil )
    end
end

def email
    if read_attribute( :email ).nil? then read_attribute( :email ) else decrypt( read_attribute( :email ), read_attribute( :email_iv ), read_attribute( :email_salt ), :email ) end
end

But when I try to read from it, it throws this OpenSSL::Cipher::CipherError: bad decrypt that more people seem to run into.

Any help would be much appreciated!

Polymorphonuclear

"that more people seem to run into" > So, what happened when you tried the solutions suggested by other threads/forums? – Viradis

Could you use these functions with test key/data and print the input/output in hexadecimals here? Print out the password, salt, IV and derived key. It's paramount that you print these values right before encryption/decryption. Note that bad decrypt could mean anything. Wrong key, wrong data, wrong encoding, wrong salt, wrong passwrod and possibly even wrong IV. – Bureaucracy

Are you sure the encodings are correct? – Breed

@owlstead that suggestion is great. I've been printing everything and it seems like the encrypted email is changed before it get's saved to db. imichaelmiers I'm pretty sure, yes, I validated that before by creating a small test script that does the encrypting and decrypting in one go, printing the string values there. Now that I think about it that too points to the problem being with the storing/reading. Duncan Jones, I tried these of course, but they didn't work for me. – Polymorphonuclear

Turned out the problem was with the Devise gem, that has case_insensitive_keys, but that triggers when saving, so when I encoded the ASCII string into UTF-8 a case sensitive string was generated, but upon save that was always lowercased, and of course if you encode that to ASCII again the result string is not going to be the same as the original one. – Polymorphonuclear

Mooi dat je het gevonden hebt, Jasper (glad you found the issue Jasper) – Bureaucracy

Answer 1

This was a bit tricky to figure out, but the problem was not with my encryption logic but with a filter in the Devise gem. Devise lowercases email addresses before save by default, but because I'm encrypting and encoding them, I'm saving case sensitive UTF8 strings to the db. Lowercased, decoding the strings back to ASCII resulted in different results than the thing before the save, and that made decryption impossible.

Now if anyone runs into this, look for the case_insensitive_keys setting in config/initializers/devise.rb, and make sure it doesn't contain the keys you are saving encrypted. Keep in mind that if you do that you'd better either lowercase the emails yourself or validate and make uppercase characters in emails prohibited, something like that.

Polymorphonuclear