When you are building a key store with the Java Keytool utility, how are the keys protected? I've read through the documentation, and I realize that each private key has a key password, and then the store has a store password.
But what mechanism is used to protect the data? Is it an encryption cipher? If so, what is the algorithm? I'm focused specifically on how keytool does the protection when it is building a JKS file.
The algorithm used depends on the keystore that you use (it could be a SmartCard, for example).
The default keystore that Sun ships with the JDK creates a soft token (on disk file) with three encryption options:
default: "jks", a proprietary keystore type (format). Not sure about the algorithm.
"jceks", an alternative proprietary format, using 3-DES
"pkcs12", a standard format (OpenSSL can read it), with several options, but usually 3-DES for private keys and RC2-40 for certificates.
In all three cases you have the private data encrypted (symmetrically, using individual passwords), and the integrity of the whole key store protected by a cryptographic digest (using the keystore password).
Sun's default JKS keystore uses a proprietary algorithm, primarily to get around exporting restrictions on standard algorithms. The algorithm is implemented in this class,
sun.security.provider.KeyProtector
The is the description of the algorithm,
This is an implementation of a Sun proprietary, exportable algorithm intended for use when protecting (or recovering the cleartext version of) sensitive keys. This algorithm is not intended as a general purpose cipher. This is how the algorithm works for key protection: p - user password s - random salt X - xor key P - to-be-protected key Y - protected key R - what gets stored in the keystore Step 1: Take the user's password, append a random salt (of fixed size) to it, and hash it: d1 = digest(p, s) Store d1 in X. Step 2: Take the user's password, append the digest result from the previous step, and hash it: dn = digest(p, dn-1). Store dn in X (append it to the previously stored digests). Repeat this step until the length of X matches the length of the private key P. Step 3: XOR X and P, and store the result in Y: Y = X XOR P. Step 4: Store s, Y, and digest(p, P) in the result buffer R: R = s + Y + digest(p, P), where "+" denotes concatenation. (NOTE: digest(p, P) is stored in the result buffer, so that when the key is recovered, we can check if the recovered key indeed matches the original key.) R is stored in the keystore. The protected key is recovered as follows: Step1 and Step2 are the same as above, except that the salt is not randomly generated, but taken from the result R of step 4 (the first length(s) bytes). Step 3 (XOR operation) yields the plaintext key. Then concatenate the password with the recovered key, and compare with the last length(digest(p, P)) bytes of R. If they match, the recovered key is indeed the same key as the original key.