.NET AES decryption breaks first few bytes

Question

I'm using AesCryptoServiceProvider to encrypt and decrypt an XML document on disk. There's an example in the MSDN reference that was helpful. I generate the AES key from the SHA-256 hash of a given password. The first half of it is assigned as IV, since I don't know of any better thing to use here. As far as I know, both key and IV must be the same for encrypting and decrypting.

When I decrypt my file, this is what the beginning of it looks like:

I���H璧�-����[�="1.0" encoding="utf-8"?>

The rest of the document is perfectly fine. There's not even some random padding after the content as I would have expected maybe.

What is causing this random garbage at the beginning of the file?

Here's more of the reading code:

using (AesCryptoServiceProvider aes = new AesCryptoServiceProvider())
{
    using (SHA256CryptoServiceProvider sha = new SHA256CryptoServiceProvider())
    {
        this.cryptoKey = sha.ComputeHash(Encoding.Unicode.GetBytes(password));
    }
    aes.Key = this.cryptoKey;
    Array.Copy(this.cryptoKey, aes.IV, 16);

    ICryptoTransform decryptor = aes.CreateDecryptor(aes.Key, aes.IV);

    using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read))
    using (CryptoStream cs = new CryptoStream(fs, decryptor, CryptoStreamMode.Read))
    using (StreamReader sr = new StreamReader(cs))
    {
        string data = sr.ReadToEnd();
        xdoc.LoadXml(data);

        //xdoc.Load(sr);
    }
}

And that's the encryption code:

XmlWriterSettings xws = new XmlWriterSettings();
xws.Encoding = Encoding.UTF8;
xws.Indent = true;
xws.IndentChars = "\t";
xws.OmitXmlDeclaration = false;

using (AesCryptoServiceProvider aes = new AesCryptoServiceProvider())
{
    aes.Key = this.cryptoKey;
    Array.Copy(this.cryptoKey, aes.IV, 16);

    ICryptoTransform encryptor = aes.CreateEncryptor(aes.Key, aes.IV);

    using (FileStream fs = new FileStream(fileName, FileMode.Create, FileAccess.Write))
    using (CryptoStream cs = new CryptoStream(fs, encryptor, CryptoStreamMode.Write))
    using (StreamWriter sw = new StreamWriter(cs, Encoding.UTF8))
    {
        XmlWriter writer = XmlWriter.Create(sw, xws);
        xdoc.Save(writer);
        writer.Close();
    }
}

Floury

Answer 1

To begin with, don't generate the key material from an ad-hoc algorithm (and yes, when it comes to key derivation, SHA256 is an ad-hoc algorithm). Follow industry standards and use a trusted Password-Based Key Derivation Function. Current standard is PBKDF-2, see also RFC2898. .Net managed crypto implementation is the Rfc2898DeriveBytes class.

Second, you must show us the encryption code. Looks to me like the sample you used appends the IV used at the beginning of the encrypted stream. Which makes perfect sense, given that the IV should not be derived from the password. The key and IV should be derived from password+random, and the 'random' must be sent as part of the file.

Officialism

Okay, I'll look into the RFC2898 thing. I just wanted to have an easy solution to use a password string instead of a bulky byte array. Also I've added the encryption code to my question. – Floury

Umm, do you have an example for that? I can't make it working. When I write salt and IV to the file and use it for decrypting, I still get random bytes instead of the XML header. – Floury

"The key and IV should be derived from password+random" > I know what you mean, but that is a little confusing to read. Perhaps instead "The key should be derived from the password and the IV should be entirely random.", or have I misunderstood you here? – Unsuccess

Okay, got it. The IV reading from the file wasn't effective. I changed it and now the file is decrypted entirely correct. Going to update my code above. Still I don't understand why using an SHA-256 as key and half of it as IV shouldn't work. It may be weaker, but after all it's only numbers and math. – Floury

@LonelyPixel Don't update the code in your question with a fix. Instead, post an answer beneath and say something like "Due to advice from Remus, I edited my code thusly and it works...". – Unsuccess

@LonelyPixel I don't understand why using an SHA-256 as key and half of it as IV shouldn't work: because you are not a cryptographer. Neither am I. this is why I simply follow industry practices. – Officialism

Answer 2

Following Remus Rusanu's advice I've changed my code to use the Rfc2898DeriveBytes class for key generation and write the used salt and IV data to the encrypted file so that I don't need to transport it on a separate channel (like the password).

This is now working for me:

// Setup XML formatting
XmlWriterSettings xws = new XmlWriterSettings();
xws.Encoding = Encoding.UTF8;
xws.Indent = true;
xws.IndentChars = "\t";
xws.OmitXmlDeclaration = false;

// Encrypt document to file
byte[] salt = new byte[8];
new RNGCryptoServiceProvider().GetBytes(salt);
Rfc2898DeriveBytes keyGenerator = new Rfc2898DeriveBytes(password, salt);

using (AesCryptoServiceProvider aes = new AesCryptoServiceProvider())
{
    aes.Key = keyGenerator.GetBytes(aes.KeySize / 8);

    using (FileStream fs = new FileStream(fileName, FileMode.Create, FileAccess.Write))
    using (CryptoStream cs = new CryptoStream(fs, aes.CreateEncryptor(), CryptoStreamMode.Write))
    using (StreamWriter sw = new StreamWriter(cs, Encoding.UTF8))
    {
        fs.Write(salt, 0, salt.Length);
        fs.Write(aes.IV, 0, aes.IV.Length);

        // Write XmlDocument to the encrypted file
        XmlWriter writer = XmlWriter.Create(sw, xws);
        xdoc.Save(writer);
        writer.Close();
    }
}

// Decrypt the file
using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read))
{
    byte[] salt = new byte[8];
    fs.Read(salt, 0, salt.Length);
    Rfc2898DeriveBytes keyGenerator = new Rfc2898DeriveBytes(this.password, salt);

    using (AesCryptoServiceProvider aes = new AesCryptoServiceProvider())
    {
        aes.Key = keyGenerator.GetBytes(aes.KeySize / 8);
        byte[] iv = new byte[aes.BlockSize / 8];
        fs.Read(iv, 0, iv.Length);
        aes.IV = iv;

        using (CryptoStream cs = new CryptoStream(fs, aes.CreateDecryptor(), CryptoStreamMode.Read))
        using (StreamReader sr = new StreamReader(cs))
        {
            // Read stream into new XmlDocument
            xdoc.Load(sr);
        }
    }
}

Floury

Thanks for reporting back with that LonelyPixel (+1). Please don't forget to accept Remus answer. – Coexecutor

+1 and thanks for posting the solution. But you shouldn't write out the IV, only the salt. The IV should be generated by the keyGenerator. Both the encryptor and the decryptor will generate the same IV. – Officialism

Remus, where should I get the IV if I don't save it to the file? The keyGenerator doesn't have a GetIV method or something. – Floury

I got it, I could just use some more bytes from keyGenerator and assign them to aes.IV both when reading and writing the file. – Floury

Answer 3

You are correct that the key and IV need to be the same for encrypting and decrypting. In many cases the IV is prepended to the cyphertext and needs to be removed before decryption. The symptom of this is extra garbage before the real start of the file. The garbage is the prepended IV.

Alternatively, you are not using the same IV, in which case the first 16 bytes of the file are garbage, and you get clean plaintext from the 17th byte onwards (AES has 16 byte blocks).

If both are happening, then you will get the first symptom.

What you have appears to be the first. Try using the first 16 bytes of the incoming file as your IV, and the rest as the actual cyphertext.

Reverent

The garbage was not prepended to the content, it was replacing the first block of the content. I thought that everybody knows how a correct XML declaration line looks like... – Floury

Thanks for pointing out my error. If it is the second symptom, then look at what IV you are using and ensure that it is byte-for-byte the same as the IV that was used for encryption. Your key is fine, given that the rest of the massage is decoding correctly. Concentrate on getting the IV right. – Reverent

Answer 4

The first block of the content is mangled because you provided the wrong IV.
The rest of the content is fine because you provided the right Key.

Clevey

Your comment was the only one that helped me solve my problem... Thank You – Defloration