views:

436

answers:

3

I'm trying to verify a file that was signed by hashing with SHA-1 and encrypting the hash with an RSA private key.

Obviously I'm using the RSA public key to verify. The key is in DER format.

The signature verification works correctly using Java's Signature class.

The openssl command I'm trying (and the result) is:

       ~/Downloads/openssl-1.0.0-beta3/apps/openssl pkeyutl -in encryptedZip.bin 
-keyform DER -verify -sigfile savedDigitalSignature.txt -pubin -inkey public.der
    WARNING: can't open config file: /usr/local/ssl/openssl.cnf
    Signature Verification Failure

I don't see anything in the openssl configuration file that would apply, so I don't think that warning is significant.

The savedDigitalSignature.txt file contains the signature bytes.

My theory is that openssl is looking for the digital signature to be in some specific file format, but I haven't found anything in the documentation indicating what that should be.

Thoughts?

A: 

I haven't used OpenSSL-1.0.0-beta3 yet, so I'm not familiar with pkeyutl, but the dgst command also generates and verifies digital signatures, and it expects signatures to be in plain, completely unadorned binary.

Eg, for an RSA signature, it expects a binary blob representing a single BigNum. Looking at the maual page for pkeyutl, it seems likely that it works the same way. An easy test would be to use pkeyutl to sign something, and inspect the output (if the size in bits is the same as your RSA key length, it's a plain BigNum - if it's bigger, try using dumpasn1 to see if its a DER format).

caf
A: 
erickson
-1: Signing and encryption with the private key are not the same thing.
Accipitridae
PKCS #1 says, "For digital signatures, the content to be signed is first reduced to a message digest with a message-digest algorithm (such as MD5), and then an octet string containing *the message digest is encrypted with the RSA private key of the signer* of the content." The difference between an RSA encryption and RSA signature is the block type prefixed to the data before encryption.
erickson
Sorry for being pedantic here. But mixing encryption and signing is a common cause for confusion and serious errors. It may also be the cause for the OPs problem. The text snippet that you cite appears to be from the 1993 version of the PKCS#1 standard. The RSA Labs have long ago noticed their mistake and none of the newer versions of the standard contain this unfortunate formulation anymore.
Accipitridae
Actually, they do. While version 2.1 does denote separate cryptographic primitives (RSASP1/RSAVP1) for signatures, in describing them, it points out that, "The main mathematical operation in each primitive is exponentiation, as in the encryption and decryption primitives of Section 5.1. RSASP1 and RSAVP1 are the same as RSADP and RSAEP except for the names of their input and output arguments; they are distinguished as they are intended for different purposes." They then proceed to duplicate the formula from the preceding section on RSADP and RSAEP.
erickson
Please read the standard carefully. Do do no longer claim that signing a message involves encryption. Anyway this is not the main point. The main observation is that some people actually use an encryption to what they think is equivalent to a signature thus potenially leading to an insecure implemenation. Hence it is important to NOT describe signing as encryption with the private key.
Accipitridae
If you can cite an authority, or even provide some concrete examples of what you are talking about, it would be much more helpful. Your critique so far has been too vague to enlighten anyone.
erickson
From your comments it is clear that you know the difference between encrypting and signing. Hence I don't understand why you insist that your answer is correct and why you don't fix it. You just use an incorrect and confusing description of what signatures are. I apologize if my comments implied anything otherwise. If you need to know why a certain padding (e.g. signatures with PKCS#1 type 2 padding) is a bad idea, then please make a new question. I can't aswer that in 4 lines.
Accipitridae
I didn't say anything about padding in my answer. The fact is that Java providers (SunJCE, BouncyCastle, and commercial providers like Entrust) use EMSA-PKCS1-V1_5 padding ("block type 1") when their RSA cipher implementation is initialized with a `PrivateKey` in `Cipher.ENCRYPT_MODE`. In this context, encryption with a `Cipher` produces results that comply with PKCS #1 v. 2.1—in particular, it produces the *same* results as a `Signature` instance (which can actually use a `Cipher` instance if the signature hash function is unspecified). In this context, it *is* correct.
erickson
Accipitridae, I certainly appreciate your input. I am a bit disappointed that you voted my question down because I misunderstood the implementation of a signature. I believe my question was specific and well-written, so an explanation of what I was misunderstanding would have been more helpful than voting it down for misunderstanding.
Wade Williams
@Wade: I didn't vote your question down. In fact, I was quite interested to know whether you generate signatures with incorrect padding by doing exactly what you describe or if your description simply was not that precise. To do so, I would have needed more input: e.g., the implementation of the signature generation, the signature itself, or the implementation of the verification. There are quite a few Java implementations that are poorly written. It may well be possible that you are using one of them.
Accipitridae
A: 

This command is very low level. You have to make sure everything is in the right format for it to work,

  1. The input signature (-sigfile) must be the binary signature. For RSA, the padding must be PKCS#1.
  2. The input data must be the binary digest. If you sign it with SHA1, this file can only contain 20 bytes.
  3. Public key must be in X.509 encoded SubjectPublicKeyInfo in DER or PEM format.
ZZ Coder