Secret vs. Non-secret Initialization Vector

Question

Today I was doing some leisurely reading and stumbled upon Section 5.8 (on page 45) of Recommendation for Pair-Wise Key Establishment Schemes Using Discrete Logarithm Cryptography (Revised) (NIST Special Publication 800-56A). I was very confused by this:

An Approved key derivation function (KDF) shall be used to derive secret keying material from a shared secret. The output from a KDF shall only be used for secret keying material, such as a symmetric key used for data encryption or message integrity, a secret initialization vector, or a master key that will be used to generate other keys (possibly using a different process). Nonsecret keying material (such as a non-secret initialization vector) shall not be generated using the shared secret.

Now I'm no Alan Turing, but I thought that initialization vectors need not be kept secret. Under what circumstances would one want a "secret initialization vector?" Thomas Pornin says that IVs are public and he seems well-versed in cryptography. Likewise with caf.

Decompound

Answer 1

An initialization vector needs not be secret (it is not a key) but it needs not be public either (sender and receiver must know it, but it is not necessary that the Queen of England also knows it).

A typical key establishment protocol will result in both involve parties computing a piece of data which they, but only they, both know. With Diffie-Hellman (or any Elliptic Curve variant thereof), the said shared piece of data has a fixed length and they have no control over its value (they just both get the same seemingly random sequence of bits). In order to use that shared secret for symmetric encryption, they must derive that shared data into a sequence of bits of the appropriate length for whatever symmetric encryption algorithm they are about to use.

In a protocol in which you use a key establishment algorithm to obtain a shared secret between the sender and the receiver, and will use that secret to symmetrically encrypt a message (possibly a very long streamed message), it is possible to use the KDF to produce the key and the IV in one go. This is how it goes in, for instance, SSL: from the shared secret (called "pre-master secret" in the SSL spec) is computed a big block of derived secret data, which is then split into symmetric keys and initialization vectors for both directions of encryption. You could do otherwise, and, for instance, generate random IV and send them along with the encrypted data, instead of using an IV obtained through the KDF (that's how it goes in recent versions of TLS, the successor to SSL). Both strategies are equally valid (TLS uses external random IV because they want a fresh random IV for each "record" -- a packet of data within a TLS connection -- which is why using the KDF was not deemed appropriate anymore).

Stairway

What he said ;). I believe the quoted text is trying to say "If you use the output of the KDF to derive the IV, you also need to ensure that the IV remains secret". In other words, treat all of the output of the KDF as a secret. – Fateful

I think the reason one might prefer to use a different "fresh random IV" with each message, is because using the same IV with every message will result in the first encrypted block of any two messages being identical if the first block of the plaintext message is identical. It is the same problem ECB encryption suffers from "The disadvantage of this method is that identical plaintext blocks are encrypted into identical ciphertext blocks; thus, it does not hide data patterns well.", and if your messages are very short(close to the size of a block), it will be nearly as easy to crack as ECB. – Takara

Regarding disk encryption, with ESSIV for example, the IV used to encrypt a disk block doesn't change. This IV is based on the key hash and disk sector. I believe that using a non-secret IV (not based on the key) for disk encryption, makes the encryption vulnerable to watermarking attack, since the IV for a particular block doesn't change. – Snaggletooth

Answer 2

Well, consider that if two parties have the same cryptographic function, but don't have the same IV, they won't get the same results. So then, it seems like the proposal there is that the two parties get the same shared secret, and each generate, deterministically, an IV (that will be the same) and then they can communicate. That's just how I read it; but I've not actually read the document, and I'm not completely sure that my description is accurate; but it's how I'd start investigating.

Girardo

Answer 3

IV is public or private, it doesn't matter let's consider IV is known to attacker, now by looking at encrypted packet/data, and knowledge of IV and no knowledge on encryption key, can he/she can guess about input data ? (think for a while)

let's go slightly backwards, let's say there is no IV in used in encryption AES (input, K)= E1 Same input will always produce the same encrypted text. Attacker can guess Key "K" by looking at encrypted text and some prior knowledge of input data(i.e. initial exchange of some protocols)

So, here is what IV helps. its added with input value , your encrypted text changes even for same input data. i.e. AES (input, IV, K)= E1 Hence, attacker sees encrypted packets are different (even with same input data) and can't guess easily. (even having IV knowledge)

Agar

Answer 4

The starting value of the counter in CTR mode encryption can be thought of as an IV. If you make it secret, you end up with some amount of added security over the security granted by the key length of the cipher you're using. How much extra is hard to say, but not knowing it does increase the work required to figure out how to decrypt a given message.

Mansfield