views:

460

answers:

4

What is the most expensive software bug / defect you've seen in practice.

There are some famous cases like the Therac 25, but I'm interested to know what issues you've seen yourself or heard of directly. We all have "a friend who used to work at an old company" and would never admit to accidently charging peoples credit cards ourselves.

What problem was caused and why was it caused?

+1  A: 

Well, I can share a story:

Long long time ago, I worked for a company that got a contract on maintaining an application that was basically written in Oracle SP's. I had to migrate one of those, and it turns out that I didn't understand the business, so I migrated it as is.

The problem was that the SP didn't do a commit at the end, but returned some data, and the migrated Java code was exactly the same. What the sp did was to write some data to a table, return a document that contained some billing information, but the billing information was never wrote to the database.

What happened next was that we received alot of payments but didn't know how to correlate them with the database data, so we had to do alot of magick to try and associate the payments.

Miguel Ping
+3  A: 

I guess this one (more info) is probably the most costly single failure (500 million USD).

Skizz

Skizz
Did you work on that project?
Dead account
No, I didn't work on that.
Skizz
+1  A: 

The Y2K bug.

flodin
Moral of the story: don't introduce arbitrary limits to facilitate micro-optimizations. Make sure your implementation can grow to handle any remotely reasonable size of values.
flodin
+1  A: 

I once shut down CNA's entire risk management system because I didn't understand what an index was on a database field. CNA approved no new Insurance policies for about four hours because the system ground to a halt. I don't know what the financial cost was, but my managers had to engage in some serious damage control.

I will never again ignore the difference between a table scan and an index seek.

Jonathan Beerhalter
+1. Ouch. Was there no UAT staging?
Dead account
Nope, no UAT, no real System Test, no staging, Nada. There was just development and live. Now that I look back, it was a pretty scary setup.
Jonathan Beerhalter