I have a Qt application written in C++ that uses a SSL-connection (QSslSocket) with another application for extra security. However, the application has a private key embedded in it.
With applications like Process Explorer it's really easy to fish out the private key. (Properties of file -> Strings)
Security is not very important for my application but it would be nice to make a little bit more difficult getting the private key from my application. Is there any way?
"strings" only finds blocks that are actual Ascii/UTF8/Unicode strings. If you keep your key around as a binary buffer then there is nothing that discriminates it from random binary data which strings usually ignores.
Other than that, There are much more clever programs out there such as IDA and OllyDebug which enable the user to fully disassemble or even decompile your program and get a hold of the key no matter what you try.
Crypt it with some simple symmetric algorithm. For example define arrays cryptedData and cryptedDataKey so that n-th byte of your private key can be get by cryptedData[cryptedDataKey[n]]. It will save you from someone who looks to your binary executable with text editor but won't help against more or less experienced person.
Also if you have persistent connections with QSslSocket a runtime it's most likely that private key is stored in memory as is. So only modifying QT library is a way to mangle key presentation in memory.
Another common technique is to put the secret data into a binary resource such as an icon image.
You may need of solutions to your problem from a different angle.
I agree with Shoosh's answer in that no matter what you do a person with the right tools and knowledge will be able to break your code and figure out your private key.
What you need to do is either externalize the data or mitigate the risks if your private keys are found.
The best way to externalize any private data is to encrypt it with a user supplied password that must be entered by the user to be used. Unfortunately this is not really reasonable for most applications.
To mitigate the risks I normally try to ensure that only the one 'install' is compromised if the security is broken.
For example, randomly generate the private key data on install.
For client/server applications you could follow the https model and use public/private key communication to exchange a randomly generated encryption key.
If each client install has there own public/private key set, then the server can tell what clients are connecting and also if there is a problem they can outlaw clients.
Hope that helps.