Should one use Cryptographic message syntax (CMS) for this task?

Question

Hello,

I've the task to transfer small binary messages (1 or 2 kb long) between a desktop application and mobile devices. The messages should be encrypted asymmetrically (RSA for instance). From what I've learned one should use a hybrid cryptosystem for this kind of task:

1. Generate random symmetric key
2. Encrypt plain text with symmetric key (using AES for instance)
3. Encrypt symmetric key with public key
4. Transmit cipher text and encrypted symmetric key

I'd like to not invent an own format for storing the cipher text and the encrypted symmetric key. So I stumbled over CMS standard (Cryptographic message syntax). At the first glance it looks exactly like what I need. If I understood the standard correctly it embeds the cipher text and the encrypted symmetric key as well as information about the used algorithms.

Can anybody say whether one should use the CMS standard for the outlined task? Does OpenSSL's CMS support is sufficient for my needs?

Cheers, Christian

Answer 1

CMS definitely supports the operation sequence you're looking for. On the downside, both the CMS format itself and the OpenSSL API for it are rather complex.

One minor wrinkle is CMS mostly operates in terms of X.509 certificates rather than public keys. You could deal with this in your system either by actually rolling out a PKI, or just using self-signed certificates (which are basically equivalent to passing around bare RSA keys, but have the advantage of being a generic format for binding the key and metadata which is sometimes quite useful to have anyway).

OpenSSL has next to no documentation of the CMS API; the best reference for it I could find is cms.c in the apps/ directory of the OpenSSL source distribution; the code is structured as one 1000 line main function, which is a little disturbing, but it does perform encryption with a public key so you can probably use that as a guide.