In standard use of asymmetrical cryptographic system, encryption is done with public key, decryption with private key.
It depends on who is doing what. Suppose Alice wants to send a message to Bob that only Bob can decode. Alice encrypts the message using Bob's public key (under the standard definition of 'public key', meaning the one that is known to people other than its owner). Now only someone who knows Bob's private key (presumably, the only person who knows Bob's private key is in fact Bob) can decrypt Alice's message to Bob.
If Alice wants Bob to know that only she could have sent it, she can encrypt the message with her own private key, assuming Bob knows her public key, either before or after encrypting the message with Bob's public key. Let's assume she encrypts the message with her private key, then the result with Bob's public key. To read the message, Bob has to decrypt the message with his (Bob's) private key, and then decrypt the result again with Alice's public key. If what he reads is now sensible text, he knows that someone who knows both Alice's private key (presumably Alice) and his public key (could be anyone at all) sent the message.
In practice, the asymmetric algorithms are expensive to compute, so what you really do is choose a random session key of an appropriate length and an agreed upon standard symmetric encryption algorithm such as AES. Then the main message is encrypted with the (relatively fast) symmetric algorithm and sent as one part of the message. The other part of the message is the encrypted - or doubly encrypted - random session key. Bob can decrypt the session key section of the message to obtain the session key; he then uses that to decrypt the main part of the message.
Note that if you are sending a message to many people, you can use one encryption of the message proper, and then encrypt the session key once for each recipient, using the recipient's public key. Each recipient can only decrypt the session key information using the key that belongs to them, but all can actually decrypt it. If the message is substantial (say 2 MB of PDF), then this is much more economical than separately encrypting the message with each recipients public key.
Inversing the process, "encryption with private key" is called "signing".
No; signing is a separate operation. If you read Schneier's "Practical Cryptography", you'll see that the authors suggest using one public/private key pair for encryption, and a second pair for signature work. For example, a signature encrypts a fixed length hash of the original message using the private key from the signing key. Anybody who knows the public key part of the signing key can then decrypt the signature to obtain the hash of the original message. Presumably, the same recipient can also decrypt the message (using the public key of the signature key pair), and then can check that the hash of the message received matches the hash derived from the signature. Any mismatch indicates a problem and the message should be discarded.
There are many ways to do these things - depending on the security requirements.
But the basic point is that one person knows the private key of an asymmetric key, and potentially many people know the public part of the asymmetric key (and this is perfectly safe). Data can be encrypted by the sender using the recipients public key; it may also be encrypted by the sender using their own private key. The recipient can decrypt the received message using their own private key and, if necessary, using the sender's public key.
The question, even as amended at about 2009-09-05T13:00-07:00, is not completely coherent, IMNSHO.
You should read chapter 13 "RSA" in "Practical Cryptography" (probably after reading some of the earlier chapters too - most notably section 3.3 Public-Key Encryption).
Notation for Encryption and Decryption
Let's define a bit of notation for discussing orthodox public key cryptography. Let's start with basic symmetric encryption:
- C = E(K,m) is the encrypted message (cipher text, C) generated by encryption algorithm E using key K on (plain text) message m.
- P = D(K,C) is the plain text message (plain text, P) discovered by decryption algorith D using key K on (encrypted) message c.
- To be a working system, m = P, so D(K,E(K,m)) = m.
So far, this notation applies to symmetric encryption because the same value K is used in both encryption and decryption. Anyone who knows K (and the algorithm, but Kerckhoff's Principle that 'secrecy is in the keys' means that you assume the attackers know the algorithm - any contrary assumption is cryptographic 'snake oil') can decrypt the message.
With an asymmetric encryption system, Ea and Da are the encryption and decryption methods for algorithm A. The key distinguishing feature of an asymmetric cryptographic cipher is that the key Kencrypt used by Ea is different from the key Kdecrypt used by Da. Further, to be practical, it must be computationally infeasible to deduce Kdecrypt even if you know Kencrypt and vice versa.
With asymmetric encryption, Alice creates a pair of keys, (Salice, Palice). Conventionally, Salice is the secret key and Palice is the public key. Note that Alice knows both keys. All that matters is:
- Salice and Palice are different.
- Alice does not let anyone else know about one of the keys (Salice); it is crucial that this information is not known to anyone else.
- Alice can let other people know about the other key (Palice) without compromising the security of the system.
Similarly, Bob will create a pair of keys, (Sbob, Pbob). Note that:
- Bob knows the keys Sbob, Pbob, and Palice.
- Alice knows the keys Salice, Palice, and Pbob.
Alice sends a message to Bob
Now, when Alice wants to send a message, Malice-bob, to Bob so that Bob can read it (but no-one else can), she has to encrypt it with Bob's key Pbob. So, she creates a message:
- Calice-bob = Ea(Pbob, Malice-bob)
Bob knows (from external evidence) that the message was encrypted with Pbob, so he knows that he must decrypt it with Sbob:
- Malice-bob = Da(Sbob, Calice-bob)
However, at this point, all he knows about the message is that it came from someone who knew his Pbob key. He does not know that it came from Alice except via extrinsic evidence.
If Bob and Alice agree that their messages must be encrypted such that they are both confident that the message received came from the other, then both must be confident that no-one other than Alice knows Salice and that no-one other than Bob knows Sbob. They must also be confident that Palice is known to Bob and Bob must be confident that Palice really does belong to Alice, and that Pbob is known to Alice and Alice must be confident that Pbob really does belong to Bob. Establishing these trust relationships is a lot of what PKI (public key infrastructure) is about.
Assuming that these criteria are met, then Alice can send her message to Bob in such a way that Bob is confident that only Alice could have sent it. As outlined previously, the mechanism is a double encryption:
- C1alice-bob = Ea(Salice,Malice-bob)
- C2alice-bob = Ea(Pbob,C1alice-bob)
Alice sends C2alice-bob to Bob (along with some signature or MAC to confirm that it was not corrupted in transit), and then Bob computes:
- D1alice-bob = Da(Sbob,C2alice-bob)
- D2alice-bob = Da(Palice,D1alice-bob)
If everything has gone according to plan, D2alice-bob = Malice-bob.
Mechanics of RSA Key Pairs
The RSA encryption algorithm is based on the fact that if you have two publicly known numbers (which are two parts of one public key), the exponent e and the modulus n, then given a message m, it is easy to compute c = me mod n. However, it is computationally infeasible to deduce m given just c (and e and n). If, however, you know another exponent, d, then you can magically calculate r = cd mod n, and r = m if you have computed e, d and n appropriately. It is not feasible to compute d from e and n without knowing some other information.
Under the RSA encryption scheme, you start work with two (large) randomly determined prime numbers, p and q, and their product is n. The RSA algorithm is predicated on the fact that it is extremely difficult to factor n (determine p and q given just n); if anyone ever finds an easy way of factoring large numbers, then the RSA algorithm is instantly broken.
Once you have n, you need to determine exponents e and d such that:
- ed = 1 mod t where t = LCM(p-1, q-1), and LCM is the least common multiple.
You can choose one of the two values as a small odd number - Schneier and Ferguson suggest e = 3, for example. You then calculate d using some computations that they cover in about 6 pages of their book. Typically, d will be a rather large number. You can then publish the pair (e, n) as the composite public key, keeping the values (p, q, t, d) secret as the private key. Given e and n, it is not computationally feasible to deduce d without first factoring n. "Practical Cryptography" suggests using two different pairs (e1, d1) and (e2, d2), derived from the same value n, where you use e1 to encrypt messages, and e2 for digital signatures; they even suggest using the values 3 and 5 for these.
OpenSSL and Key Generation
Your description of how the RSA keys are generated by OpenSSL is confused, I believe.
The generation process first has to generate to large random prime numbers, p and q in the notation above. There are stochastic methods for determining whether a given large number is (probably) prime; it takes a little while to compute two such prime numbers. Taken together, these are used to compute first n, and then d (assuming e is established by some convention). The two stages you see in OpenSSL are determining n, and then determining d.
Dissection of User Case
The question says:
Consider that Alice wants to send to Bob some stuff in a non-traditional way:
Alice and Bob once met and Alice gave Bob a "public key" generated from a private key she created BUT she warned Bob to keep it secret. AND she kept secret the private key, and didn't ever give to anyone else the public key.
So far, so good. The 'public key' isn't very public, but there's no harm in that.
Could Bob be sure that messages he receives from Alice (provided these are encrypted by Alice private key) are only readable by him (provided he really kept his copy of Alice's public key secret)?
If the encryption technology is of any use, then yes; only Alice and Bob can read the message that Alice encrypted with her secret key because only Alice and Bob know the public key that goes with her secret key.
And how compares this encryption solidity to the traditional way, which would, in our case, be Bob sending messages (encrypted by the public key of Alice) to Alice?
Confusion: the section started by discussing Alice sending messages to Bob; now you've switched to Bob sending messages to Alice.
When Bob and Alice met, Alice gave Bob her Palice public key. Presumably, Bob also gave Alice his Pbob public key. And both public keys have very limited public circulation - that's good, but not crucial to the security of the system.
Now, when Bob wants to send a message to Alice, he can encrypt it with her Palice public key, and Alice (and only Alice) can decrypt the message using her Salice secret key. Alternatively, Bob could encrypt the message with his Sbob secret key, and Alice could decrypt it with Bob's Pbob public key. Both sets of encryption and decryption would work.
What the question is about
The fact that asymmetrical keys are named "private" and "public" doesn't help understanding my question. Keys have underlying properties, and it's me broadcasting the "public key" that gives it its "public" property. Please make this distinction clear before answering: I'm not considering the "public" and "private" properties of these keys but the solidity of the "private key" encryption versus "public key" encryption.
It is equally reliable to encrypt with the correct private key and decrypt with the correct public key as it is to encrypt with the correct public key and decrypt with the correct private key. The difference is in who can do which operation. If you understand clearly who is doing the encrypting and who is doing the decrypting, and who knows which keys, then the secrecy of the methods become fairly clear.
I cannot use another terminology even if it is misleading in this special case.
Well, the 'public keys' in your case are not all that widely known, but that's all that's unusual about it.
I know that this case is non-traditional, and could lead to several inconsistency, or is not the point of the asymmetrical crypto systems as Bob and Alice here share some sort of a common secret and that's not the point of asymmetrical crypto.
The whole point of asymmetric encryption schemes is that it does not matter whether the attackers (classically called Eve, the eavesdropper) knows the public key. As long as the private keys are kept private by Alice and Bob, the messages can be sent securely. However, you must understand that if Alice sends a message to Bob that is encrypted only by Alice's secret key, then anyone (such as Eve) who knows Alice's public key can read the message. Eve can't create a fake message that purports to come from Alice unless she also knows the secret key - if Eve discovers Alice's secret key, Eve can pretend to be Alice at any time she likes. But she can read it. If Alice sends a message to Bob that is encrypted only by Bob's public key, then only Bob can read the message (using his secret key), but Bob has no way of knowing whether it actually came from Alice or whether Eve sent it pretending to be Alice. That's why you have to work hard to ensure that Bob knows that only Alice could have sent the message, and Alice knows that only Bob can read the message.