What are the standard encryption file formats?

Question

I'm a bit confused on encryption file formats.

Let's say I want to encrypt a file with AES-256. I run the file through the encryption algorithm and I now have a stream of encrypted bytes.

I obviously can write that stream of bytes to a file, but any third-party encryption application is not going to understand it since it's not expecting just a raw stream of encrypted bytes.

Into what file formats can I write that so that other encryption tools can understand it?

The ones I know of (I think) are:

PKCS#7
ASN.1
DER
PEM
PKCS#8

but I'm not sure how they all relate to each other.

Apparently the AESCrypt utility also has a format, which appears to be its own proprietary format: http://www.aescrypt.com/aes%5Ffile%5Fformat.html

Is there a cheatsheet anywhere on this stuff? I've been googling and found bits and pieces, but never felt like I had the complete picture.

Answer 1

AES is an encryption algorithm, not a file format.

As you point out, there are lots of knobs and levers on the algorithm - key strength is one. AES-256 just means, the AES algorithm with 256-bit key. But there are lots of other knobs. Mode, for one. AES has a number of modes: CBC, ECB, OFB, CFB, CTR, and others. Another is the IV, which applies to some modes. Padding is another. Usually these knobs are exposed in the AES api for whatever framework you're using.

In most cases AES is combined with other crypto technology - for example password-based key derivation (PBKDF2) is often used to generate keys or IVs. MAC's are often used to verify the integrity of the encrypted data.

Different tools use AES to encrypt, and if they want their data to be readable, they publish the list of knobs they use, and how they are set, as well as how any related crypto technology might be used.

When creating a file format, you'll need to store or publish those kinds of things, if you want your file to be readable by other applications.

Answer 2

PKCS#8 is not an encrypted-file format, it's a format for private keys.

ASN.1 and DER are rules for translating a structured message into binary. They are not, in and of themselves, a file format, although they're used to define and describe file formats.

PKCS#7 is closely related to PEM, and they're both formats for public-key encrypted files. They are defined in terms of base-64 encapsulated DER encoded ASN.1 messages. They are the basis of the S/MIME format for secure internet mail. (see RFC3851)

In parallel with S/MIME is the OpenPGP file format, also mainly designed for public-key encrypted files. (See RFC4880)

In both S/MIME and OpenPGP formats, there is a block which contains symmetric-key encrypted data. It is possible to create valid S/MIME or OpenPGP files containing only this block. In this way, the S/MIME (a.k.a. PKCS#7) and OpenPGP formats can be used for symmetric-key encryption also.