What are the standard encryption file formats?

Question

I'm a bit confused on encryption file formats.

Let's say I want to encrypt a file with AES-256. I run the file through the encryption algorithm and I now have a stream of encrypted bytes.

I obviously can write that stream of bytes to a file, but any third-party encryption application is not going to understand it since it's not expecting just a raw stream of encrypted bytes.

Into what file formats can I write that so that other encryption tools can understand it?

The ones I know of (I think) are:

PKCS#7
ASN.1
DER
PEM
PKCS#8

but I'm not sure how they all relate to each other.

Apparently the AESCrypt utility also has a format, which appears to be its own proprietary format: http://www.aescrypt.com/aes_file_format.html

Is there a cheatsheet anywhere on this stuff? I've been googling and found bits and pieces, but never felt like I had the complete picture.

Hidie

Answer 1

PKCS#8 is not an encrypted-file format, it's a format for private keys.

ASN.1 and DER are rules for translating a structured message into binary. They are not, in and of themselves, a file format, although they're used to define and describe file formats.

PKCS#7 is closely related to PEM, and they're both formats for public-key encrypted files. They are defined in terms of base-64 encapsulated DER encoded ASN.1 messages. They are the basis of the S/MIME format for secure internet mail. (see RFC3851)

In parallel with S/MIME is the OpenPGP file format, also mainly designed for public-key encrypted files. (See RFC4880)

In both S/MIME and OpenPGP formats, there is a block which contains symmetric-key encrypted data. It is possible to create valid S/MIME or OpenPGP files containing only this block. In this way, the S/MIME (a.k.a. PKCS#7) and OpenPGP formats can be used for symmetric-key encryption also.

Chartreuse

What about PKCS#12 standard? It defines an archive file format, could that be use as a standard encrypted file format for both symmetric and asymmetric encryption? However I see that it seems to require a password, but doesn't mention asymmetric encryption. – Webfoot

@Webfoot - while it's theoretically possible (i.e. valid according to the spec) to store arbitrary data in a PKCS#12 (see RFC7292) file, it is uncommon to use it that way. The original question was about interoperability with other applications, and this would not be a highly interoperable format. – Chartreuse

@Webfoot However, if you do want to store arbitrary data in PKCS12, it's similar to S/MIME, in that the content is a PKCS#7 (see RFC2315) ContentInfo blob encrypted with a symmetric key, and in the asymmetric / public key scenario, the symmetric key is randomly generated, encrypted with the public key, and stored alongside the content. The RFCs describe this as "Enveloped". – Chartreuse

Answer 2

AES is an encryption algorithm, not a file format.

As you point out, there are lots of knobs and levers on the algorithm - key strength is one. AES-256 just means, the AES algorithm with 256-bit key. But there are lots of other knobs. Mode, for one. AES has a number of modes: CBC, ECB, OFB, CFB, CTR, and others. Another is the IV, which applies to some modes. Padding is another. Usually these knobs are exposed in the AES api for whatever framework you're using.

In most cases AES is combined with other crypto technology - for example password-based key derivation (PBKDF2) is often used to generate keys or IVs. MAC's are often used to verify the integrity of the encrypted data.

Different tools use AES to encrypt, and if they want their data to be readable, they publish the list of knobs they use, and how they are set, as well as how any related crypto technology might be used.

When creating a file format, you'll need to store or publish those kinds of things, if you want your file to be readable by other applications.

Brunella

Right - but aren't there standards-based file formats? So that a program opening the file can discover the algorithm, the parameters used, etc? That's what I'm asking - are there formats into which I can write encrypted data, and expect another program to be able to understand it (assuming it implements that algorithm in the same way, the user has been given the key or the public key included, etc). If such formats exist, where can I find a summary of my options? Or is it always "roll your own?" Thanks for the help. – Hidie

Well, sure. For example WinZip can use AES, and it documents how it does so. winzip.com/aes_info.htm PKCS#7 is documented in RFC 2315. tools.ietf.org/html/rfc2315 MS-Word uses AES encryption. download.microsoft.com/download/6/7/f/… I'm not sure if you have a specific format in mind, or .. you want to know about lots of formats, or what. – Brunella

Answer 3

You might want to look into Crypt4GH which was standardized at the end of 2019.

Crypt4GH, a new standard file container format from the Global Alliance for Genomics and Health (GA4GH), allows genomic data to remain secure throughout their lifetime, from initial sequencing to sharing with professionals at external organizations.

From what I can see it is similar - in terms of crypto - to NaCl's crypto_box, but with the advantage of formalizing a file format on disk.

Grassland

Answer 4

JSON Web Encryption RFC 7516 is an IETF standard that can do what you are looking for, it can handle AES in addition to other crypto algorithms.

JSON Web Encryption (JWE) represents encrypted content using JSON-
based data structures [RFC7159]. The JWE cryptographic mechanisms
encrypt and provide integrity protection for an arbitrary sequence of octets.

There implementing of JWE in multiple languages for example in Java you can use nimbus

Leadership