Initialization vector - best practices (symmetric cryptography)

Question

I would like to ask about best practices regarding a usage of an initialization vector (IV) and a key for symmetric cryptography algorithms.

I want to accept messages from a client, encrypt them and store in a backend. This will be done over a time, and there will be requests coming at a later time for pooling out the messages and return them in a readable form.

According what I know, the key can be the same during the encryption of multiple separate messages. The IV should change with every new encryption. This however, will cause problems, because every message will need a different IV for de-cryption at a later time.

I’d like to know if this is the best way of doing it. Is there any way to avoid storing IV with every message, which would simplify entire process of working with encryption/decryption?

Kampala

Instead of generating randomly, use IV starting from 0 and increment by one for every message. This is common practice than randomly generating and guarantees the distinction. – You

There would be a problem when obtaining data from the database. The returned records from searching according unencrypted index field, will contain rows encrypted with IV in non-incremental order: instead of 1,2,3,... IV can be 1, 10, 35 .. – Kampala

The IV can be the unique row ID? – You

Do you want to query over the encrypted data? – You

@You A counter is ok for modern modes, but not for CBC (bad) or CTR (catastrophically bad). Using a unique row ID as the IV is not good in general because row IDs can be reused. – Clough

Answer 1

IV selection is a bit complicated because the exact requirements depend on the mode of operation. There are some general rules, however:

• You can't go wrong¹ with a random IV, except when using shorter IVs in modes that allow this.
• Never use the same IV with the same key.
• If you only ever encrypt a single message with a given key, the choice of IV doesn't matter².
• Choose the IV independently of the data to encrypt.
• Never use ECB.

Of the most common specific modes of operation:

• CBC requires the IV to be generated uniformly at random. Do not use a counter as IV for CBC. Furthermore, if you're encrypting some data that contains parts that you receive from a third party, don't reveal the IV until you've fully received the data, .
• CTR uses the IV as the initial value of a counter which is incremented for every block, not for every message, and the counter value needs to be unique for every block. A block is 16 bytes for all modern symmetric ciphers (including AES, regardless of the key size). So for CTR, if you encrypt a 3-block message (33 to 48 bytes) with 0 as the IV, the next message must start with IV=3 (or larger), not IV=1.
• Modern modes such as Chacha20, GCM, CCM, SIV, etc. use a nonce as their IV. When a mode is described as using a nonce rather than an IV, this means that the only requirement is that the IV is never reused with the same key. It doesn't have to be random.

When encrypting data in a database, it is in general not safe to use the row ID (or a value derived from it) as IV. Using the row ID is safe only if the row is never updated or removed, because otherwise the second time data is stored using the same ID, it would repeat the IV. An adversary who sees two different messages encrypted with the same key and IV may well be able to decrypt both messages (the details depend on the mode and on how much the attacker can guess about the message content; note that even weak guesses such as “it's printable UTF-8” may suffice).

Unless you have a very good reason to do otherwise (just saving a few bytes per row does not count as a very good reason) and a cryptographer has reviewed the specific way in which you are storing and retrieving the data:

• Use an authenticated encryption mode such as GCM, CCM, SIV or Chacha20+Poly1305.

• If you can store a counter somewhere and make sure that it's never reset as long as you keep using the same encryption key, then each time you encrypt a message:

  1. Increment the counter.
  2. Use the new value of the counter as the nonce for the authenticated encryption.

  The reason to increment the counter first is that if the process is interrupted, it will lead to a skipped counter value, which is not a problem. If step 2 was done without step 1, it would lead to repeating a nonce, which is bad. With this scheme, you can shave a few bytes off the nonce length if the mode allows it, as long as the length is large enough for the number of messages that you'll ever encrypt.

• If you don't have such a counter, then use the maximum nonce length and generate a random counter. The reason to use the maximum nonce length is that due to the birthday paradox, a random n-bit nonce is expected to repeat when the number of messages approaches 2^n/2.

• In either case, you need to store the nonce in the row.

¹ _{Assuming that everything is implemented correctly, e.g. random values need to be generated with a random generator that is appropriate for cryptography.}
² _{As long as it isn't chosen in a way that depends on the key.}

Patently