Finding Vulnerabilities in Software

Question

Hello,

I'm insterested to know the techniques that where used to discover vulnerabilities. I know the theory about buffer overflows, format string exploits, ecc, I also wrote some of them. But I still don't realize how to find a vulnerability in an efficient way.

I don't looking for a magic wand, I'm only looking for the most common techniques about it, I think that looking the whole source is an epic work for some project admitting that you have access to the source. Trying to fuzz on the input manually isn't so comfortable too. So I'm wondering about some tool that helps.

E.g.

I'm not realizing how the dev team can find vulnerabilities to jailbreak iPhones so fast. They don't have source code, they can't execute programs and since there is a small number of default programs, I don't expect a large numbers of security holes. So how to find this kind of vulnerability so quickly?

Thank you in advance.

Answer 1

19 Deadly Sins of Software Security is a good book.

Jeff Atwood also blogged about it.

Answer 2

On the lower layers, manually examining memory can be very revealing. You can certainly view memory with a tool like Visual Studio, and I would imagine that someone has even written a tool to crudely reconstruct an application based on the instructions it executes and the data structures it places into memory.

On the web, I have found many sequence-related exploits by simply reversing the order in which an operation occurs (for example, an online transaction). Because the server is stateful but the client is stateless, you can rapidly exploit a poorly-designed process by emulating a different sequence.

As to the speed of discovery: I think quantity often trumps brilliance...put a piece of software, even a good one, in the hands of a million bored/curious/motivated people, and vulnerabilities are bound to be discovered. There is a tremendous rush to get products out the door.

Answer 3

Aside from buffer overflow and format string exploits, you may want to read a bit on code injection. (a lot of what you'll come across will be web/DB related, but dig deeper) AFAIK this was a huge force in jailbreaking the iThingies. Saurik's mobile substrate allow(s) (-ed?) you to load 3rd party .dylibs, and call any code contained in those.

Answer 4

Using genetic alogorithm

http://stackoverflow.com/questions/3790959/genetic-algorithms-for-computer-security

Google tech conference about revese engineering patch updates :-)

http://www.youtube.com/watch?v=mwrhRP2PswA