views:

1626

answers:

11

Report released today on the "top 25" most dangerous programming mistakes. I'm interested to see if any here agree or can spot any glaring omissions (or outdated inclusions).

Also, in your opinion which modern dev tools/frameworks are improving (or worsening) these flaws for things like authorization, SQL injection, or code injection?

The List

  1. Improper Input Validation
  2. Improper Encoding or Escaping of Output
  3. Failure to Preserve SQL Query Structure (aka 'SQL Injection')
  4. Failure to Preserve Web Page Structure (aka 'Cross-site Scripting')
  5. Failure to Preserve OS Command Structure (aka 'OS Command Injection')
  6. Cleartext Transmission of Sensitive Information
  7. Cross-Site Request Forgery (CSRF)
  8. Race Condition
  9. Error Message Information Leak
  10. Failure to Constrain Operations within the Bounds of a Memory Buffer
  11. External Control of Critical State Data
  12. External Control of File Name or Path
  13. Untrusted Search Path
  14. Failure to Control Generation of Code (aka 'Code Injection')
  15. Download of Code Without Integrity Check
  16. Improper Resource Shutdown or Release
  17. Improper Initialization
  18. Incorrect Calculation
  19. Improper Access Control (Authorization)
  20. Use of a Broken or Risky Cryptographic Algorithm
  21. Hard-Coded Password
  22. Insecure Permission Assignment for Critical Resource
  23. Use of Insufficiently Random Values
  24. Execution with Unnecessary Privileges
  25. Client-Side Enforcement of Server-Side Security
+1  A: 

It seems to me that a lot of these are very closely related or are pretty much the same thing. XSS is similar to the one about not encoding correctly...

Joe Philllips
A lot of contemporary security flaws are caused by violating a few simple principles; commonly, "don't trust user input".
Rob
also similar to code injection...
annakata
+2  A: 

While the list seems to be a great check list for a code review, I think the underlying cause for most of these is the same: Under-motivated programmers in a hostile environment.

See for example this Joel article about how to make wrong code look wrong to see how the environment (variable naming guidelines in this particular example) can help avoiding serious security holes.

David Schmitt
or under-supported...
annakata
annakata++, hmemcpy++ (though I'd count both points under hostile environment)
David Schmitt
+1  A: 

In my view the list is obvious, redunant in some respects and omits important line items.

CWE-116 is mearly a high-level superset of CWE-89, CWE-79, CWE-78 and CWE-94.

Do CWE-73 and CWE-426 really need to be separate line items?

Do CWE-404 and CWE-665 really need to be separate line items?

Input validation (CWE-20) is absoultely not a security issue. Its a GIGO issue provided you don't violate CWE-116 when processing said input.

No mention of management of encryption keys - extremely important, very easy to get wrong.

No mention of identity/authentication/trust - "encryption" is meaningless without it.

Einstein
+12  A: 

The main set of tools and frameworks that is making these worse is beginner-oriented coding forums where people routinely post example code for newbies that contains serious vulnerabilities. In particular, examples that are vulnerable to SQL injection are absolutely rampant.

I speak out against this when I see it, and on one such occasion I was even yelled at by several forum regulars because they believed that writing PHP examples that wrap inputs in mysql_real_escape_string() was "too complex" for beginners and they should be taught to consider security at some later date. The brain-damage simply astounds.

chaos
Yes, or in general, bad teaching, whether oral or written - teach someone to do something the wrong way, and they'll do it the wrong way until someone else teaches them the right way. Possibly with the use of a big stick.
Rob
What's worst is that even examples in frameworks documentation are written with the same mistakes. (For example I remember some ADO.NET examples that concatenates strings to obtain the SQL)
Aleris
Devil's advocate here: I think that's OK. It's like when you have a line, "Window *myWindow; //Assume this has already been set". If setting that variable isn't relevant to the example, the extra code is just distracting. The problem is people who turn their brains off when they see example code.
Chuck
@Chuck: If it were a situation of leaving out where the variable gets set, that would be marginally acceptable. It's not. The scenarios I, and I believe Aleris, are talking about explicitly show setting and using variables in a vulnerable way. Which is not okay, at all.
chaos
@Chuck: Teach them the right thing from the start, then teach them again and again. I have mentored so many developers over the years and it astounds me that even experienced developers make fundamental mistakes because no one corrected them!
Student for Life
+10  A: 

I think they forgot "Exposing your application to users", winner of top risk for at least 30 years running.

krosenvold
Of course, the real problem with software development is the users. It's unbelievable. They've caused problems with every program I've ever written.
Jeff Atwood
It'll be on the list next year, together with incorrect programming of "lost" recordings in TIVO.
krosenvold
+2  A: 
  10. 'Failure to Constrain Operations within the Bounds of a Memory Buffer'

Thus StackOverflow is bad for programming... I've gotta expand my memory buffer. Hehe.

null
+4  A: 

They seem to miss "Improper Exception usage", where an unchecked exception somewhere make another portion of code not executed because that exception breaks the control flow.

(See Disclaimer at the end of this post: what follows is not "against" exceptions, it is just a reminder that exceptions can be tricky ;) )

Joel has viewed Exception as "Goto in disguise" (October 2003), and explained in his "Making Wrong Code Look Wrong" (may 2005) that:

In order to make code really, really robust, when you code-review it, you need to have coding conventions that allow collocation.

In other words, the more information about what code is doing is located right in front of your eyes, the better a job you’ll do at finding the mistakes.

When you have code that says

dosomething();
cleanup();

… your eyes tell you, what’s wrong with that? We always clean up!

But the possibility that dosomething might throw an exception means that cleanup might not get called. And that’s easily fixable, using finally or whatnot, but that’s not my point:
my point is that the only way to know that cleanup is definitely called is to investigate the entire call tree of dosomething to see if there’s anything in there, anywhere, which can throw an exception, and that’s ok, and there are things like checked exceptions to make it less painful, but the real point is that exceptions eliminate collocation.

You have to look somewhere else to answer a question of whether code is doing the right thing, so you’re not able to take advantage of your eye’s built-in ability to learn to see wrong code, because there’s nothing to see.


Disclaimer: this "programming error" I mention here is not "for" or "against" exceptions (I fully agree with using Exception, and I also use "error code" when needed).
It merely points out that writing good exception-based code is hard.
As Raymond Chen pointed out:

when I write code that is exception-based, I do not have the luxury of writing bad code first and then making it not-bad later. If I did that, I wouldn't be able to find the bad code again, since it looks almost identical to not-bad code.
My point isn't that exceptions are bad. My point is that exceptions are too hard and I'm not smart enough to handle them


I am puzzled by the comments which imply that Joel's arguments are borderline "specious" (i.e. "apparently good or right though lacking real merit"):

Greg Beech: "always use finally for clean up, and it's probably right."
Richard Corden: "even if you don't explicitly throw an exception yourself - you might get an exception you don't expect (eg. allocating memory)"
Andreas Huber: "Even if dosomething() does not throw any exception today it might well tomorrow, so why not make things robust and add the finally in any case???? P.S. A little Google research would show you that pretty much everything Joel mentions w.r.t. exceptions has long been debunked"

Andreas cite this article "Lessons Learned from Specifying Exception-Safety for the C++ Standard Library" as a first example of rebuttal.

I fail to see how those comments contradict in any way:

  • Joel's point: exceptions eliminate collocation
  • Raymond's point: exceptions are too hard and I'm not smart enough to handle them.

For one, there are many ways to handle exceptions improperly: look at those "Exception Horror Stories" which, in themselves, should grand "Exception Management" a nice spot in that "most dangerous programming mistakes" list.

And then, there is the issue of "just using finally". "Just"... Right. As if it was simple or trivial, as if the result was still readable.
Consider this "Good housekeeping practices" article from IBM.

What is a readable code (below) is a bad, unsafe one, because exceptions are thrown:

public void enumerateFoo() throws SQLException {
    Connection connection = getConnection();
    Statement statement = connection.createStatement();
    ResultSet resultSet = statement.executeQuery("SELECT * FROM Foo");
    // Use resultSet
    resultSet.close();
    statement.close();
    connection.close();
}

"Just add a finally" does not solve anything, while adding complexity:

public void enumerateFoo() throws SQLException {
    Statement statement = null;
    ResultSet resultSet = null;
    Connection connection = getConnection();
    try {
        statement = connection.createStatement();
        resultSet = statement.executeQuery("SELECT * FROM Foo");
        // Use resultSet
    }
    finally {
        if (resultSet != null) {
            resultSet.close(); }
        if (statement != null) {
            statement.close(); }
        connection.close();
    }

}

Using finally to release resources acquired in a method is reliable but can easily get unwieldy when multiple resources are involved.

The reason this "solution" doesn't work is that the close() methods of ResultSet and Statement can themselves throw SQLException, which could cause the later close() statements in the finally block not to execute.
That leaves you with several choices, all of which are annoying:

  • wrap each close() with a try..catch block,
  • nest the try...finally blocks
  • or write some sort of mini-framework for managing the resource acquisition and release.

The "nesting solution" would be:

public void enumerateBar() throws SQLException {
    Statement statement = null;
    ResultSet resultSet = null;
    Connection connection = getConnection();
    try {
        statement = connection.createStatement();
        resultSet = statement.executeQuery("SELECT * FROM Bar");
        // Use resultSet
    }
    finally {
        try {
            if (resultSet != null) {
                resultSet.close(); }
        }
        finally {
            try {
                if (statement != null) {
                    statement.close(); }
            }
            finally {
                connection.close();
            }
        }
    }
}

Aaargh... (and I do that in my code...).
Compare the above with the initial lines:

[...]
ResultSet resultSet = statement.executeQuery("SELECT * FROM Foo");
// Use resultSet
resultSet.close();
[...]

Do you see now how "exceptions eliminate collocation" ?

This business of making wrong code look wrong depends on getting the right things close together in one place on the screen: [...] get the relevant information about what a line of code really does physically as close together as possible. This improves the chances that your eyeballs will be able to figure out everything that’s going on.

With all those "null checks" and "nested finally", keeping code together is kind of hard...

VonC
"the only way to know that cleanup is definitely called is to investigate the entire call tree of dosomething" ? How about: try { dosomething(); } finally { cleanup(); } . No need to "investigate the entire call tree of dosomething"
Andreas Huber
@Andreas Huber: how about "that’s easily fixable, using finally or whatnot": Joel mentioned it! And he also mentionned it is not his point. His point is: before adding the finally you suggest, you have to INVESTIGATE if dosomething throws any unchecked exception. Hence the "eliminate collocation".
VonC
@VonC - There's a strong likelihood that anything which needs to be cleaned up could raise an error during allocation or initialization, as usually anything that can be cleaned up is a scarce resource that can failed to be allocated. So always use finally for clean up, and it's probably right.
Greg Beech
@Greg Beech: very true, but again, THAT IS NOT JOEL's POINT. Consider a code where you do not realize "cleanup()" is actually a consequence from "dosomething()"... you will be bitten by an unchecked exception. Please read his article. He is on to something ;)
VonC
The issue with Joel's point above, is that (at least in C++) even if you don't explicitly throw an exception yourself - you might get an exception you don't expect (eg. allocating memory). The solution is to use RAII for the cleanup code (ie. a destructor is responsible for cleaning up).
Richard Corden
@Richard Corden: true but I think Joel was not always referring to initialization/cleanup scenario, but to any kind of 2 blocs of code, the second dependent of the first. In absolute, exception can break that execution dependency, even if no 'new' memory alloc are involved.
VonC
@VonC: Yes he does mention adding finally and then *still* complains that he has to investigate the whole call tree before doing that. Why would he want to investigate the call tree in the first place? Even if dosomething() does not throw any exception today it might well tomorrow, (continued...)
Andreas Huber
... so why not make things robust and add the finally in any case???? Which brings us to my point: The code dosomething(); cleanup(); is simply broken in any language that allows for exceptions. If someone has written it so that a reader doesn't realize that cleanup() must always be (continued ...)
Andreas Huber
... called then that someone has simply committed a huge coding blunder. So, the problem entirely lies with the author of the code and not the exception mechanism.P.S. A little Google research would show you that pretty much everything Joel mentions w.r.t. exceptions has long been debunked.
Andreas Huber
@Andreas Huber: interesting, I will do the search, but in the meantime, do not hesitate to share some links in those comments ;)
VonC
@VonC: http://www.boost.org/community/exception_safety.html. The article argues about exception safety in C++ and most of the insights are transferable to other languages. However, some problems (e.g. throwing copy operations) do not exist in other languages, which makes things easier.
Andreas Huber
@Andreas Huber: I am still not sure you can ever "debunk" Joel's "collocation" argument: This business of making wrong code look wrong depends on getting the right things close together in one place on the screen, and with all those nested "finally", that can be hard to do... See my completed answer
VonC
The first book I ever read using the ideas of cohesion and coupling blasted cleanup routines as having high coupling and low cohesion. I was taken aback by that, but eventually realized that any function named cleanup() is almost certainly a mistake.
David Thornley
The "mini-framework for managing the resource acquisition and release" is spelled RAII in C++. It's a fundamental concept. Learn it or please find a job with a competitor of my company. Exception safety isn't inherently hard, but often taught badly.
David Thornley
@David Thornley: thank you for your feedback. [Humor on] Since I am not smart (http://stackoverflow.com/questions/251007#251066), I am looking for a competitor of your company right away ;).
VonC
@VonC: The unwieldy nested finally-blocks can be avoided by defining that resource release must never throw an exception. The following article has the details for C++: http://stackoverflow.com/questions/130117/throwing-exceptions-out-of-a-destructor. (continued)
Andreas Huber
An exception thrown from a resource release method when the method is called as a result of another exception, has different results in different languages:- C++ unceremoniously terminates your process. The agreed solution in C++ is simply "don't throw from a destructor". (continued)
Andreas Huber
- .NET unceremoniously replaces the first exception with the second. So, there's *some* consensus that you shouldn't throw from a Dispose() method, because you'd otherwise have fewer clues why your process went down- Don't know about Java, but if there isn't a similar rule there better should be.
Andreas Huber
P.S. The second version of your enumerateBar() function is just fine and IMO still readable. BTW, in C#, thanks to the using keyword, the enumerateBar() function could be written in just 5 lines, with correct exception handling!
Andreas Huber
A: 

I like the list.

nes1983
A: 

The OWASP Top Ten list is probably more focussed - but SAN's list (when did SAN become the NSA?) adds some useful bits.

blowdart
+6  A: 

The list seems to have been written from a very narrow point of view, namely security-related mistakes. It also seems pretty web-app-centric. They don't address mistakes that relate to:

  • correctness and stability
  • performance
  • maintainability

IMO all those fields require just as much attention as security; there are many apps that don't really need security at all.

One example: while race conditions can be a security problem, they're much more important in regard to correctness and stability - the standard example is the Therac-25, a radiation therapy machine that, due to a race condition, killed several people.

Michael Borgwardt
+1: note that it's a lot easier to assure security when the app is correct and maintainable.
David Thornley
A: 

I agree with the list but also believe that publishing these types of lists help with popularity but don't help people write better code. I would encourage developers to pay more attention to OWASP in this regard.

jm04469