Finding security problems in a given code

Question

Hello guys,

Can some one please tell me an approach for finding security flaws in a given code. For ex: in a given socket program. Any good examples or good book recommendations are welcome.

Thanks & Regards,

Mousey

Answer 1

I took a security class where we used a commercial product called Fortify 360, which did static analysis of C++ code. We ran it against an old-old-old version of OpenSSL, and it found loads of stuff, and provided guidance to rectify the flaws (which, by the way, the latest version of OpenSSL had resolved).

At any rate, it is a useful commercial tool.

Answer 2

The lowest hanging fruit in this category would be to simply search the source for functions which are commonly misused or are difficult use safely such as:

• strcpy
• strcat
• sprintf
• gets

then start looking at ones that are not inherintly too bad, but could be misused. Particularly anything that writes to a buffer can potentially be hazardous if misused.

• memcpy
• memmove
• recv/read
• send/write
• the entire printf family should always have a constant for the format string

NOTE: all of these (except gets) can be used correctly, so don't think it's a flaw just because the function is used, instead take a look at how it is used. Also note that gets is always a flaw.

NOTE2: this list is not exhaustive, do a little research about commonly misused functions and how they can be avoided.

As far as tools, I recommend things like valgrind and splint

Answer 3

Here's a book recommendation: Writing Secure Code. Demonstrates not only how to write secure code, but also common pitfalls and practices that expose security holes. It's slightly dated (my copy says it was published in 2002), but the security concepts it teaches are still quite applicable even 8 years later.

Answer 4

One major topic that wasn't covered in Evan's answer is integer overflows. Here are some examples:

wchar_t *towcs(const char *s)
{
    size_t l = strlen(s)+1;
    mbstate_t mbs = {0};
    wchar_t *w = malloc(l*sizeof *w), *w2;
    if (!w || (l=mbsrtowcs(w, (char **)&s, l, &st))==-1) {
        free(w);
        return 0;
    }
    return (w2=realloc(w, l*sizeof *w)) ? w2 : w;
}

Here, a giant string (>1gig on 32-bit) will make multiplication by the size (I'm assuming 4) overflow, resulting in a tiny allocation and subsequent writes past the end of it.

Another more common example:

uint32_t cnt;
fread(&cnt, 1, 4, f);
cnt=ntohl(cnt);
struct record *buf = malloc(cnt * sizeof *buf);

This sort of code turns up in reading file/network data quite a lot, and it's subject to the same sort of overflows.

Basically, any arithmetic performed on values obtained from an untrusted source, which will eventually be used as an allocation size/array offset, needs to be checked. You can either do it the cheap way (impose arbitrary limits on the value read that keep it significantly outside the range which could overflow, or you can test for overflow at each step: Instead of:

foo = malloc((x+1)*sizeof *foo);

You need to do:

if (x<=SIZE_MAX-1 && x+1<=SIZE_MAX/sizeof *foo) foo = malloc((x+1)*sizeof *foo);
else goto error;

A simple grep for malloc/realloc with arithmetic operators in its argument will find many such errors (but not ones where the overflow already occurred a few lines above, etc.).

Answer 5

Some source code constructs you can keep an eye out for are:

• Functions that don't do bounds checking. Evan covered it pretty well.
• Input validation & sanitization, or lack thereof.
• NULL pointer dereferencing
• fork()s, execve()s, pipe()s, system() called with non-static parameters (or worse, with user input).
• Objects shared between threads with inappropriate storage durations (pointers to automatic variables or even "dead" objects in thread-local storage).
• When dealing with file manipulation, make sure correct variable types are used for the return results of functions. Make sure they're checked for errors. Make no assumptions about the implementation - permissions of created files, uniqueness of filenames, etc.
• Poor sources of randomness (for encryption, communication, etc.) should be avoided.
• Simple or obvious mistakes (perhaps out of carelessness) should be fixed anyway. You never know what's exploitable, unless it is.

Also, are the data protected? Well, if you don't care, that's fine. :-)

Some tools that you can consider are:

• valgrind : exposes memory flaws, which in large applications are usually critical.
• splint : a static checker
• fuzzing frameworks
• RATS : a free, open-source tool. Its authors' company was acquired by Fortify.

Answer 6

Some of the OpenBSD folk just recently published a presentation on their coding practices.