Good AES Initialization Vector practice

Question

per my question Aes Encryption... missing an important piece, I have now learned that my assumption for creating a reversible encryption on a string was a bit off. I now have

    public static byte[] EncryptString(string toEncrypt, byte[] encryptionKey)
    {
        var toEncryptBytes = Encoding.UTF8.GetBytes(toEncrypt);
        using (var provider = new AesCryptoServiceProvider())
        {
            provider.Key = encryptionKey;
            provider.Mode = CipherMode.CBC;
            provider.Padding = PaddingMode.PKCS7;
            using (var encryptor = provider.CreateEncryptor(provider.Key, provider.IV))
            {
                using (var ms = new MemoryStream())
                {
                    using (var cs = new CryptoStream(ms, encryptor, CryptoStreamMode.Write))
                    {
                        cs.Write(toEncryptBytes, 0, toEncryptBytes.Length);
                        cs.FlushFinalBlock();
                    }
                    return ms.ToArray();
                }
            }
        }
    }

and this produces consistent results; however, I will not be able to decrypt without knowing/ setting the initialization vector. I really do not want to pass three values into this method (on for the IV), which leaves me with hardcoding the IV or deriving it from the key. I'd like to know if this is a good practice, or if it will render the encrypted value vulnerable to attack somehow... or am I really overthinking this and should just hardcode the IV?

UPDATE Per Iridium's suggestion, I tried something like this instead:

    public static byte[] EncryptString(string toEncrypt, byte[] encryptionKey)
    {
        if (string.IsNullOrEmpty(toEncrypt)) throw new ArgumentException("toEncrypt");
        if (encryptionKey == null || encryptionKey.Length == 0) throw new ArgumentException("encryptionKey");
        var toEncryptBytes = Encoding.UTF8.GetBytes(toEncrypt);
        using (var provider = new AesCryptoServiceProvider())
        {
            provider.Key = encryptionKey;
            provider.Mode = CipherMode.CBC;
            provider.Padding = PaddingMode.PKCS7;
            using (var encryptor = provider.CreateEncryptor(provider.Key, provider.IV))
            {
                using (var ms = new MemoryStream())
                {
                    ms.Write(provider.IV, 0, 16);
                    using (var cs = new CryptoStream(ms, encryptor, CryptoStreamMode.Write))
                    {
                        cs.Write(toEncryptBytes, 0, toEncryptBytes.Length);
                        cs.FlushFinalBlock();
                    }
                    return ms.ToArray();
                }
            }
        }
    }

    public static string DecryptString(byte[] encryptedString, byte[] encryptionKey)
    {
        using (var provider = new AesCryptoServiceProvider())
        {
            provider.Key = encryptionKey;
            provider.Mode = CipherMode.CBC;
            provider.Padding = PaddingMode.PKCS7;
            using (var ms = new MemoryStream(encryptedString))
            {
                byte[] buffer;
                ms.Read(buffer, 0, 16);
                provider.IV = buffer;
                using (var decryptor = provider.CreateDecryptor(provider.Key, provider.IV))
                {
                    using (var cs = new CryptoStream(ms, decryptor, CryptoStreamMode.Read))
                    {
                        byte[] decrypted = new byte[encryptedString.Length];
                        var byteCount = cs.Read(decrypted, 0, encryptedString.Length);
                        return Encoding.UTF8.GetString(decrypted, 0, byteCount);
                    }
                }
            }
        }
    }

however, this shows something odd in my unit test:

    [TestMethod]
    public void EncryptionClosedLoopTest()
    {
        var roundtrip = "This is the data I am encrypting.  There are many like it but this is my encryption.";
        var encrypted = Encryption.EncryptString(roundtrip, encryptionKey);
        var decrypted = Encryption.DecryptString(encrypted, encryptionKey);
        Assert.IsTrue(roundtrip == decrypted);
    }

my decrypted text shows up as "92ʪ�F"�,hpv0�� I am encrypting. There are many like it but this is my encryption." which seems almost right but of course completely wrong. It looks like I'm close though. Am I missing an offset on the memory stream?

Hsining

For whatever reason, you can't edit bytes of the IV, you have to set the whole IV byte[]. In DecryptString(), you need to read the IV from the stream into a new byte[], and then set the provider.IV to this new byte[]. – Mauricemauricio

In your decryption function, you aren't accounting for the offset of the IV when calling cs.Read(). Better to just use a StreamReader instead. e.g. using (var sr = new StreamReader(cs)) { return sr.ReadToEnd(); } – Cockhorse

@Cockhorse I'm not clear on what you mean. If you see an improvement to make, can you provide an answer? – Hsining

@JeremyHolovacs I found that ms.Read(provider.IV, 0, 16); is not correctly populating the IV property. Therefore I created a new byte[] buffer and filled that from the stream. Then set provider.IV = ivBuffer; – Neurosurgery

@Sam, I'm not sure what you mean; my unit tests passed cleanly and I have used this code in production for several years now. What is "correctly" in this context? – Hsining

@JeremyHolovacs that is strange. I started an empty project and copy/pasted most of the code and I could not get the code to properly decrypt. I found that "ms.Read(provider.IV, 0, 16);" was not populating the "IV" property correctly. – Neurosurgery

@Neurosurgery it turns out you're right, when I went back and looked at it I ended up using a buffer for it as well. – Hsining

Answer 1

The IV should be random and unique for every run of your encryption method. Deriving it from the key/message or hard-coding it is not sufficiently secure. The IV can be generated within this method, instead of passed into it, and written to the output stream prior to the encrypted data.

When decrypting, the IV can then be read from the input before the encrypted data.

Mauricemauricio

Can you explain why? What value does this provide to maintain 2 pieces of arbitrary data (key and IV) that need to be referenced for encryption and decryption vs. 1 (just the key)? – Hsining

The key protects the encrypted data, whereas the use of a random IV ensures that information is not leaked by the ciphertext itself. IT does this by preventing identical plaintexts from producing identical ciphertext when encrypted using the same key. – Mauricemauricio

I followed your advice and attached the initialization vector to the byte array. – Hsining

I just found out, to decrypt, you have to use same IV otherwise first few chars will be scrambled. Then.. if you use random and unique IV, lets say generated by GenerateIV();, then to be able to decrypt the string, you have to store IV with it together. Ok, I have to say the result looks nice, encrypted string changes all the time. But do you think store(or even transfer) IV with encrypted string together is too risky and also too much of work? Or I misunderstand some code here, that I do not need to use same IV to encrypt and decrypt. – Chromatics

@EricYin Yes, you need to use the same IV for decrypting as was used to encrypt. There is no risk to transmitting the IV in the clear along with the encrypted message. As to whether it's too much work, that's up to you, but it will be less secure if you don't. – Mauricemauricio

FWIW OpenSSL's key derivation function derives both a key and an IV based on a password and a salt. An advantage of this approach is now you can use a fixed salt size regardless of the algorithm used (now handled behind the scenes) – Being

@Mauricemauricio Thank you I think I finally get the use of IVs now. That info combined with a comment I read last night saying IV is similar to salt so mainly to protect against rainbow tables has made it all make sense. – Person

Answer 2

When Encrypting, generate your IV and pre-pend it to the cipher text (something like this)

        using (var aes= new AesCryptoServiceProvider()
        {
            Key = PrivateKey,
            Mode = CipherMode.CBC,
            Padding = PaddingMode.PKCS7
        })
        {

            var input = Encoding.UTF8.GetBytes(originalPayload);
            aes.GenerateIV();
            var iv = aes.IV;
            using (var encrypter = aes.CreateEncryptor(aes.Key, iv))
            using (var cipherStream = new MemoryStream())
            {
                using (var tCryptoStream = new CryptoStream(cipherStream, encrypter, CryptoStreamMode.Write))
                using (var tBinaryWriter = new BinaryWriter(tCryptoStream))
                {
                    //Prepend IV to data
                    //tBinaryWriter.Write(iv); This is the original broken code, it encrypts the iv
                    cipherStream.Write(iv);  //Write iv to the plain stream (not tested though)
                    tBinaryWriter.Write(input);
                    tCryptoStream.FlushFinalBlock();
                }

                string encryptedPayload = Convert.ToBase64String(cipherStream.ToArray());
            }

        }

When decrypting this back, get first 16 bytes out and use it in crypto stream

var aes= new AesCryptoServiceProvider()
                    {
                        Key = PrivateKey,
                        Mode = CipherMode.CBC,
                        Padding = PaddingMode.PKCS7
                    };

                    //get first 16 bytes of IV and use it to decrypt
                    var iv = new byte[16];
                    Array.Copy(input, 0, iv, 0, iv.Length);

                    using (var ms = new MemoryStream())
                    {
                        using (var cs = new CryptoStream(ms, aes.CreateDecryptor(aes.Key, iv), CryptoStreamMode.Write))
                        using (var binaryWriter = new BinaryWriter(cs))
                        {
                            //Decrypt Cipher Text from Message
                            binaryWriter.Write(
                                input,
                                iv.Length,
                                input.Length - iv.Length
                            );
                        }

                        return Encoding.Default.GetString(ms.ToArray());
                    }

Chaetognath

I am using DES algorithm – Huygens

Doesn't tBinaryWriter.Write(iv); get encrypted in the stream rather than get prepended in "plaintext"? – Dissolvent

Daniel Park is correct. I've tested the code in this answer and it produces the SAME OUTPUT every time for the same data. Sadly, I can't remove my upvote. – Chowchow

I have updated the bug in the source code and hopefully this is works better – Hf

I know this answer is really old, but why do we need to do aes.GenerateIV();? Doesn't new AesCryptoServiceProvider() generate a key for you? – Throne

using return Encoding.Default.GetString(ms.ToArray()); may lead to weird characters being shown on your decrypted string. I'd recommend using Convert.ToBase64String() and Convert.FromBase64String – Christcrossrow

Answer 3

Great input from folks. I took the combined answers from ankurpatel and Konstantin and cleaned it up and added some convenient method overrides. This works as of June 2019 in .NET Core 2.2.

using System;
using System.IO;
using System.Security.Cryptography;
using System.Text;

private const int AesKeySize = 16;

public static void Main()
{
    // the data to encrypt
    var message = "Here is some data to encrypt!";

    // create KeySize character key
    var key = "g(KMDu(EEw63.*V`";

    // encrypt the string to a string
    var encrypted = AesEncrypt(message, key);

    // decrypt the string to a string.
    var decrypted = AesDecrypt(encrypted, key);

    // display the original data and the decrypted data
    Console.WriteLine($"Original:   text: {encrypted}");
    Console.WriteLine($"Round Trip: text: {decrypted}");
}

static string AesEncrypt(string data, string key)
{
    return AesEncrypt(data, Encoding.UTF8.GetBytes(key));
}

static string AesDecrypt(string data, string key)
{
    return AesDecrypt(data, Encoding.UTF8.GetBytes(key));
}

static string AesEncrypt(string data, byte[] key)
{
    return Convert.ToBase64String(AesEncrypt(Encoding.UTF8.GetBytes(data), key));
}

static string AesDecrypt(string data, byte[] key)
{
    return Encoding.UTF8.GetString(AesDecrypt(Convert.FromBase64String(data), key));
}

static byte[] AesEncrypt(byte[] data, byte[] key)
{
    if (data == null || data.Length <= 0)
    {
        throw new ArgumentNullException($"{nameof(data)} cannot be empty");
    }

    if (key == null || key.Length != AesKeySize)
    {
        throw new ArgumentException($"{nameof(key)} must be length of {AesKeySize}");
    }

    using (var aes = new AesCryptoServiceProvider
    {
        Key = key,
        Mode = CipherMode.CBC,
        Padding = PaddingMode.PKCS7
    })
    {
        aes.GenerateIV();
        var iv = aes.IV;
        using (var encrypter = aes.CreateEncryptor(aes.Key, iv))
        using (var cipherStream = new MemoryStream())
        {
            using (var tCryptoStream = new CryptoStream(cipherStream, encrypter, CryptoStreamMode.Write))
            using (var tBinaryWriter = new BinaryWriter(tCryptoStream))
            {
                // prepend IV to data
                cipherStream.Write(iv);
                tBinaryWriter.Write(data);
                tCryptoStream.FlushFinalBlock();
            }
            var cipherBytes = cipherStream.ToArray();

            return cipherBytes;
        }
    }
}

static byte[] AesDecrypt(byte[] data, byte[] key)
{
    if (data == null || data.Length <= 0)
    {
        throw new ArgumentNullException($"{nameof(data)} cannot be empty");
    }

    if (key == null || key.Length != AesKeySize)
    {
        throw new ArgumentException($"{nameof(key)} must be length of {AesKeySize}");
    }

    using (var aes = new AesCryptoServiceProvider
    {
        Key = key,
        Mode = CipherMode.CBC,
        Padding = PaddingMode.PKCS7
    })
    {
        // get first KeySize bytes of IV and use it to decrypt
        var iv = new byte[AesKeySize];
        Array.Copy(data, 0, iv, 0, iv.Length);

        using (var ms = new MemoryStream())
        {
            using (var cs = new CryptoStream(ms, aes.CreateDecryptor(aes.Key, iv), CryptoStreamMode.Write))
            using (var binaryWriter = new BinaryWriter(cs))
            {
                // decrypt cipher text from data, starting just past the IV
                binaryWriter.Write(
                    data,
                    iv.Length,
                    data.Length - iv.Length
                );
            }

            var dataBytes = ms.ToArray();

            return dataBytes;
        }
    }
}

Carsoncarstensz

Great, but change Encoding.Default to Encoding.UTF8, see learn.microsoft.com/en-us/dotnet/api/… for details. – Sounder

You rock. To make it work for me in net48 I had to call cipherStream.Write with the buffer length: cipherStream.Write(iv, 0, iv.Length). I just replaced some old code where a dev hardcoded the IV.. fun. – Typewriting

I have fixed Paultechguy's solution with Nothingisnecessary's solution for .NET 4.8 and it works for AesKeySize = 16 but it doesn't work for AesKeySize = 32 because the key generated is always 16 so how can I fix it for AesKeySize = 32? – Paleethnology

For .NET4.8 and 256bits encryption, change private const int AesKeySize = 16; by private const int AesKeySize = 32; and var iv = new byte[AesKeySize]; by var iv = new byte[aesAlgo.BlockSize / 8]; in the AesDecrypt method and it will work for 32 bytes encryption key. – Paleethnology

Answer 4

The accepted answer is correct, but doesn't provide a good example of how to get a random IV.

It turns out that this is a lot easier than people are trying to make it. The AesCryptoServiceProvider in .NET automatically generates a cryptographically random IV every time you construct one. And if you need to use the same instance for multiple encryptions, you can call GenerateIV()

You can also prepend the IV to the encrypted value before returning it and have the decrypting end pull it off

private static void Main(string[] args) {
    var rnd = new Random(); 
    var key = new byte[32];  // For this example, I'll use a random 32-byte key.
    rnd.NextBytes(key);
    var message = "This is a test";

    // Looping to encrypt the same thing twice just to show that the IV changes.
    for (var i = 0; i < 2; ++i) {
        var encrypted = EncryptString(message, key);
        Console.WriteLine(encrypted);
        Console.WriteLine(DecryptString(encrypted, key));
    }
}

public static string EncryptString(string message, byte[] key) {
    var aes = new AesCryptoServiceProvider();
    var iv = aes.IV;
    using (var memStream = new System.IO.MemoryStream()) {
        memStream.Write(iv, 0, iv.Length);  // Add the IV to the first 16 bytes of the encrypted value
        using (var cryptStream = new CryptoStream(memStream, aes.CreateEncryptor(key, aes.IV), CryptoStreamMode.Write)) {
            using (var writer = new System.IO.StreamWriter(cryptStream)) {
                writer.Write(message);
            }
        }
        var buf = memStream.ToArray();
        return Convert.ToBase64String(buf, 0, buf.Length);
    }
}

public static string DecryptString(string encryptedValue, byte[] key) {
    var bytes = Convert.FromBase64String(encryptedValue);
    var aes = new AesCryptoServiceProvider();
    using (var memStream = new System.IO.MemoryStream(bytes)) {
        var iv = new byte[16];
        memStream.Read(iv, 0, 16);  // Pull the IV from the first 16 bytes of the encrypted value
        using (var cryptStream = new CryptoStream(memStream, aes.CreateDecryptor(key, iv), CryptoStreamMode.Read)) {
            using (var reader = new System.IO.StreamReader(cryptStream)) {
                return reader.ReadToEnd();
            }
        }
    }  
}

[EDIT: I modified my answer to include how to pass the IV in the encrypted value and get it when decrypting. I also refactored the example a bit]

Aloysia

This does not address the question. – Hsining

@JeremyHolovacs It sort of is, but I see your point too. He was asking about how to get the IV and said he didn't want to hard code it or get it from the key. Iridum's answer said it should be unique but didn't say how to get one, hence my answer. But what he really was asking was how to get the IV later for decryption. I'll modify my answer to address that. – Aloysia

to be clear, OP was written by me, and I was honestly trying to understand what the IV was for (and what it got me)... before this, I had assumed it was like some kind of secondary key, which seemed superflous. After this, I understood it to be more akin to a salt in a hashing algorithm. I'm also not clear on how this is significantly "simpler", but at least the answer addresses the question so I'll remove my downvote. – Hsining

Answer 5

I modified your decryption method as follows and it works:

public static string DecryptString(byte[] encryptedString, byte[] encryptionKey)
{
    using (var provider = new AesCryptoServiceProvider())
    {
        provider.Key = encryptionKey;
        using (var ms = new MemoryStream(encryptedString))
        {
            // Read the first 16 bytes which is the IV.
            byte[] iv = new byte[16];
            ms.Read(iv, 0, 16);
            provider.IV = iv;

            using (var decryptor = provider.CreateDecryptor())
            {
                using (var cs = new CryptoStream(ms, decryptor, CryptoStreamMode.Read))
                {
                    using (var sr = new StreamReader(cs))
                    {
                        return sr.ReadToEnd();
                    }
                }
            }
        }
    }
}

The problem with your implementation is that you are reading too many bytes into the CryptoStream. You really need to read encryptedText.Length - 16. Using a StreamReader simplifies this, since you don't need to worry about offsets anywhere anymore.

Cockhorse

StreamReader.ReadToEnd() treats zeros as end of message and is therefore unusable for general data. BinaryReader should be used instead. – Workingwoman

Answer 6

In order to resolve the setting of IV on the provider (As Iridium pointed out):

ms.Read(provider.IV, 0, 16);

I added the following to your code:

var iv = new byte[provider.IV.Length];
memoryStream.Read(iv, 0, provider.IV.Length);
using (var decryptor = provider.CreateDecryptor(key, iv);

granted, my key is not set by the provider on each run. I generated it once and then stored it. The IV is randomly generated off of the provider for each encryption.

Bloomers

Answer 7

Actually I have doubt here 

string initVector = "HR$2pIjHR$2pIj12";//instead of this line I need a single line change
byte[] initVectorBytes = Encoding.ASCII.GetBytes(initVector);
_iv = Convert.ToBase64String(initVectorBytes);

I need to get IV here but I need to pass initVector as dynamic. How to pass it?

Midyear

This looks like it should be a question, not an answer. – Hsining

Answer 8

In my case, to generate the IV, I use something like this

    /// <summary>
    /// Derives password bytes
    /// </summary>
    /// <param name="Password">password</param>
    /// <returns>derived bytes</returns>
    private Rfc2898DeriveBytes DerivePass(string Password)
    {
        byte[] hash = CalcHash(Password);
        Rfc2898DeriveBytes pdb = new Rfc2898DeriveBytes(Password, hash, _KEY_ITER);
        return pdb;
    }

    /// <summary>
    /// calculates the hash of the given string
    /// </summary>
    /// <param name="buffer">string to hash</param>
    /// <returns>hash value (byte array)</returns>
    private byte[] CalcHash(string buffer)
    {
        RIPEMD160 hasher = RIPEMD160.Create();
        byte[] data = Encoding.UTF8.GetBytes(buffer);
        return hasher.ComputeHash(data);
    }

that is, I calculate the password hash using RIPEMD160 and use it to generate the derived bytes, at that point, when it comes to intializing the encryption/decryption I just use something like this

        Rfc2898DeriveBytes pdb = DerivePass(Password);
        SymmetricAlgorithm alg = _engine;
        alg.Key = pdb.GetBytes(_keySize);
        alg.IV = pdb.GetBytes(_IVSize);

I don't know if it's "correct" (probably crypto gurus here will shoot at me :D), but, at least, it gives me a decent IV and I don't have to store it "somewhere" since just entering the correct password will give back the needed IV value; as a note, the _engine in the above example is declared as "SymmetricAlgorithm" and initialized using something like this

_engine = Rijndael.Create();
_keySize = (_engine.KeySize / 8);
_IVSize = (_engine.BlockSize / 8);

which creates the desired crypto objects and initializes the key and IV sizes

Hypothecate

This makes your IV always the same for the same password which weakens the encryption. – Aloysia

Answer 9

To generate random IV you would need a truly random number. Whichever language specific API you use for generating the random number, should generate true random number. Both android and ios have apis which generate random numbers based on sensor data.

I recently implemented AES 256 with random IV (Generated using really random numbers) and hashed key. For more secure(random IV + hashed key) cross platform (android, ios, c#) implementation of AES see my answer here - https://mcmap.net/q/275652/-aes-gets-different-results-in-ios-obj-c-and-android-java

Cellini

The CryptoProvider already has a function to generate a random IV. – Aloysia