Decrypting CryptoStream into MemoryStream

Question

I have written a process where a file is encrypted and uploaded to Azure, then the download process has to be decrypted which is what fails with a "Padding is invalid and cannot be removed" error, or a "Length of the data to decrypt is invalid." error.

I've tried numerous solutions online, including C# Decrypting mp3 file using RijndaelManaged and CryptoStream, but none of them seem to work and I end up just bouncing back and forth between these two errors. The encryption process uses the same key/IV pair that decryption uses, and since it will decrypt a portion of the stream I feel like that's working fine - it just ends up dying with the above errors.

Here is my code, any ideas? Please note that the three variants (cryptoStream.CopyTo(decryptedStream), do {} and while) aren't run together - they are here to show the options I've already tried, all of which fail.

byte[] encryptedBytes = null;

using (var encryptedStream = new MemoryStream())
{
    //download from Azure
    cloudBlockBlob.DownloadToStream(encryptedStream);

    //reset positioning for reading it back out
    encryptedStream.Position = 0;

    encryptedBytes = encryptedStream.ConvertToByteArray();
}

//used for the blob stream from Azure
using (var encryptedStream = new MemoryStream(encryptedBytes))
{
    //stream where decrypted contents will be stored
    using (var decryptedStream = new MemoryStream())
    {
        using (var aes = new RijndaelManaged { KeySize = 256, Key = blobKey.Key, IV = blobKey.IV })
        {
            using (var decryptor = aes.CreateDecryptor())
            {
                //decrypt stream and write it to parent stream
                using (var cryptoStream = new CryptoStream(encryptedStream, decryptor, CryptoStreamMode.Read))
                {
                    //fails here with "Length of the data to decrypt is invalid." error
                    cryptoStream.CopyTo(decryptedStream);

                    int data;

                    //fails here with "Length of the data to decrypt is invalid." error after it loops a number of times,
                    //implying it is in fact decrypting part of it, just not everything
                    do
                    {
                        data = cryptoStream.ReadByte();
                        decryptedStream.WriteByte((byte)cryptoStream.ReadByte());
                    } while (!cryptoStream.HasFlushedFinalBlock);

                    //fails here with "Length of the data to decrypt is invalid." error after it loops a number of times,
                    //implying it is in fact decrypting part of it, just not everything
                    while ((data = cryptoStream.ReadByte()) != -1)
                    {
                        decryptedStream.WriteByte((byte)data);
                    }
                }
            }
        }

        //reset position in prep for reading
        decryptedStream.Position = 0;
        return decryptedStream.ConvertToByteArray();
    }
}

One of the comments mentioned wanting to know what ConvertToByteArray is, and it's just a simple extension method:

/// <summary>
/// Converts a Stream into a byte array.
/// </summary>
/// <param name="stream">The stream to convert.</param>
/// <returns>A byte[] array representing the current stream.</returns>
public static byte[] ConvertToByteArray(this Stream stream)
{
    byte[] buffer = new byte[16 * 1024];
    using (MemoryStream ms = new MemoryStream())
    {
        int read;
        while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
        {
            ms.Write(buffer, 0, read);
        }
        return ms.ToArray();
    }
}

The code never reaches this though - it dies before I can ever get it to this point.

Schaal

does DownloadToStream work correctly? have you checked it? What is ConvertToByteArray? – Dis

Yep, that is fine - it's only the decryption part that doesn't work. I only put that in to show where encryptedBytes comes from, which is just the download from Azure. I updated the code to show the extension for ConvertToByteArray, though it never gets to that. – Schaal

RubyHaus. As you may already know, new questions get more attentions and more votes. Therefore people are reluctant to answer stale questions. (Deleteting this one and asking it again isn't a solution too, people hate it) Good luck. – Dis

How is this stale? I asked it less than 24 hours ago. – Schaal

Answer 1

After a lot of back and forth from various blogs, I found I actually had a couple of errors in the above code that were nailing me. First, the encryption process was incorrectly writing the array - it was wrapped with a CryptoStream instance, but wasn't actually utilizing that so I was writing the unencrypted data to Azure. Here is the proper route to go with this (fileKey is part of a custom class I created to generate Key/IV pairs, so wherever that is referenced can be changed to the built-in process from RijndaelManaged or anything else you'd utilize for coming up with a key/IV pair):

using (var aes = new RijndaelManaged { KeySize = 256, Key = fileKey.Key, IV = fileKey.IV })
{
    using (var encryptedStream = new MemoryStream())
    {
        using (ICryptoTransform encryptor = aes.CreateEncryptor())
        {
            using (CryptoStream cryptoStream = new CryptoStream(encryptedStream, encryptor, CryptoStreamMode.Write))
            {
                using (var originalByteStream = new MemoryStream(file.File.Data))
                {
                    int data;
                    while ((data = originalByteStream.ReadByte()) != -1)
                        cryptoStream.WriteByte((byte)data);
                }
            }
        }

        var encryptedBytes = encryptedStream.ToArray();
        return encryptedBytes;
    }
}

Second, since my encryption process involves multiple steps (three total keys per file - container, filename and file itself), when I tried to decrypt, I was using the wrong key (which is seen above when I referenced blobKey to decrypt, which was actually the key used for encrypting the filename and not the file itself. The proper decryption method was:

//used for the blob stream from Azure
using (var encryptedStream = new MemoryStream(encryptedBytes))
{
    //stream where decrypted contents will be stored
    using (var decryptedStream = new MemoryStream())
    {
        using (var aes = new RijndaelManaged { KeySize = 256, Key = blobKey.Key, IV = blobKey.IV })
        {
            using (var decryptor = aes.CreateDecryptor())
            {
                //decrypt stream and write it to parent stream
                using (var cryptoStream = new CryptoStream(encryptedStream, decryptor, CryptoStreamMode.Read))
                {
                    int data;

                    while ((data = cryptoStream.ReadByte()) != -1)
                        decryptedStream.WriteByte((byte)data);
                }
            }
        }

        //reset position in prep for reading
        decryptedStream.Position = 0;
        return decryptedStream.ConvertToByteArray();
    }
}

I had looked into the Azure Encryption Extensions (http://www.stefangordon.com/introducing-azure-encryption-extensions/), but it was a little more local file-centric than I was interested - everything on my end is streams/in-memory only, and retrofitting that utility was going to be more work than it was worth.

Hopefully this helps anyone looking to encrypt Azure blobs with zero reliance on the underlying file system!

Schaal

Is doing it byte by byte inefficient? Any reason why you can't use CopyTo ? – Heretofore

@rolls - you are right. In the final using statement, one can use cryptoStream.CopyTo(decryptedStream); instead of writing byte by byte. – Showpiece

Answer 2

Bit late to the party, but in case this is useful to someone who finds this thread:

The following works well for me.

internal static byte[] AesEncryptor(byte[] key, byte[] iv, byte[] payload)
    {
        using (var aesAlg = Aes.Create())
        {
            aesAlg.Mode = CipherMode.CBC;

            aesAlg.Padding = PaddingMode.PKCS7;

            var encryptor = aesAlg.CreateEncryptor(key, iv);

            var encrypted = encryptor.TransformFinalBlock(payload, 0, payload.Length);

            return iv.Concat(encrypted).ToArray();
        }
    }

and to decrypt:

internal static byte[] AesDecryptor(byte[] key, byte[] iv, byte[] payload)
    {
        using (var aesAlg = Aes.Create())
        {
            aesAlg.Mode = CipherMode.CBC;

            aesAlg.Padding = PaddingMode.PKCS7;

            var decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV);

            return decryptor.TransformFinalBlock(payload, 0, payload.Length);
        }
    }

this works for encrypting/decrypting both fixed length hex strings when decoded from hex to byte[] as well as utf8 variable length strings when decoded using Encoding.UTF8.GetBytes().

Tollhouse

This method was about twice as fast as the answers that used the memorystream / cryptostream technique. Here's my stopwatch data. | Time Old Way: 0:00.052 Time New Way: 0:00.024| Time Old Way: 0:00.046 Time New Way: 0:00.032| |Time Old Way: 0:00.054 Time New Way: 0:00.024| |Time Old Way: 0:00.051 Time New Way: 0:00.029| |Time Old Way: 0:00.064 Time New Way: 0:00.028| |Time Old Way: 0:00.059 Time New Way: 0:00.035| – Amylopectin

This is bad practice; if you want to implement manually, you should use TransformBlock for all but the Final Block. However, output will be the same for AES. AES Blocks are 16Bytes. – Zobkiw