What is the biggest problem with software development?

Question

What do you think the biggest problem is with software development?

To me, when I think of the problems I face developing software, I think of a lack of precision. There's too much "Hey, let's try this out and maybe it'll work."

Not all of you may work in a field that requires any kind of precision, but for me, I'm constantly battling the lack of specifications, and the lack of documentation. If I screw something up, lots of money is going to be lost, but the process that we (and most software development places, I think) use basically forces me to fudge my work. I have to skim over some important things to make sure something gets done in time, or skip some important bug checking.

I think there's just too much focus on "getting it out the door" that we lose focus on what we're trying to do: Create something that does its job well, and requires a minimum of maintaining.

How can we fix this? Well, I do my best by trying to shore up the lack of specs and documentation. I write my own specs where there aren't enough, keep a strict bug checking policy, and try to be as careful as possible. Something will always slip through the cracks though, and until The Powers can accept that GOOD software takes a lot of time to develop and test, I don't think we'll have it much easier.

Note: This is just my experience with my field. Your mileage may vary.

So, what do you think is the most pervasive problem in our industry?

Answer 1

There are really two problems: bad programmers and bad project managers.

Answer 2

Our biggest problem, in my opiinion and experience, is simply that business refuses to accept our proposed timelines - and we suffer continually for it.

YMMV.

Answer 3

I'd say the biggest problem is long cycle times. Ask yourself these questions about a current project of yours:

1. How long does it take for me to compile and test a change?
2. How long does it take for a (coding-wise, simple) concept to go from brain to code to repository?
3. How long does it take for a bug to go from reported to fixed?
4. If you've come up with a list of 20 things that might improve X, assuming each of those things is trivial to code, how long does it take you to test all 20 things?

I am always amazed how these numbers vary between companies and projects. In some cases, one can find a problem, fix it, and commit the result in 10 minutes. In other cases, I've heard stories of utterly trivial fixes taking weeks to percolate through bureaucracy, management, and version control, slowing down everything else depending on them.

This is also why some small open source projects can wipe the floor with commercial competition--if the commercial version is so utterly bogged down by incompetent management and bad methodology, a single coder in his free time can sometimes make a better product than an entire professional development team. Such is the power of bad organization and slow cycle times.

Of course, open source projects don't inherently have short cycle times, either--some of the larger projects can be plagued by exactly the same problems as large closed-source projects.

Answer 4

Microsoft. No, seriously. My wife doesn't understand that I don't mind that Bill Gates is rich, I mind that he got rich peddling crap. People have come to expect that software sucks. Is it Donald Norman who noted how depressing it is that toddlers know to fix things by turning them off and back on? If consumers had higher standards for acceptable systems -- for personal computing or embedded systems -- there would be more pressure for quality and less pressure for fast releases and systems would be better.

Answer 5

The lack of appreciation for what it takes to build quality software

Answer 6

Ever Rising Expectations

Take Stack Overflow. It was put together in a couple of months with a couple of people, most of them working part time.

That would have been incredible just a couple of years ago, and now we take it for granted.

Examples abound in all areas of programming.

Answer 7

Delivering what the client needs, not what the client seems to say that he/she needs. Cutting through misconceptions and miscommunication from everyone involved, me included.

Answer 8

The pace of advancement. When you're building a bridge, you can rely on knowing stuff that has been known for 100 years - best practices, what to make the bridge out of, etc, how big a load it can hold, etc. But if, every year, a new material for bridge building came out, and the rivers got wider, and the bridge had to carry twice as many people, you'd see the same problems we have in software engineering. Stuff is changing too fast.

Not that that's a bad thing... :)

Answer 9

The single biggest problem is the lack of understanding about what software development actually is: both on the part of software developers and on the part of the managers that are leading the development effort. Particularly with regard to quality/time/feature tradeoffs...and that these tradeoffs are very fuzzy and usually vary based on the particular developers that are doing the work.

There are these implicit metaphors that lie under the surface of the thinking of many managers and even a lot of software developers, such as:

1) Developing software is like baking cookies: you put the right ingredients in, give it a specific amount of time in the oven and you get exactly what you want. Wrong!

2) It's Possible to know, up-front, exactly what a system should and will look like. Wrong!

3) Users know what they need. Wrong!

Not many managers / developers will state these outright - but many believe them, and their actions show it.

So - if you're a developer you should be doing your best to learn the craft as well as possible (not just the programming - but also estimating, managing business requirements, etc) and you should educate your leadership about the unique challenges that face software development when it comes to predictability.

If you're a mangager you should realize that software development is like no other endeavor that you'll ever have the opportunity to lead - it is not like assembling a car or baking cookies - and bringing predictability to it is a skill that not many folks possess.

A good resource for both managers and developers is: "Behind Closed Doors" by Rothman and Derby. Excellent, easy read and very helpful.

Answer 10

The never ending Fast (Time), Cheap (Money), Good (Quality) triangle discussion. Probably should update my approach with Susan Warrens stool analogy :-)

Answer 11

There are two problems with Software Development and both originate on the business side but are exacerbated by our reactions.

The first is that companies are in a struggle for survival so mistakes carry definite costs that can impact more than simply those making them. This can cause companies to go on periodic witch hunts to find out "why did that happen!?!" If you are in a company prone to these witch hunts, it is easy to develop CYA habits that take a toll on what you are able to accomplish. This is most often the source of the irritating catch 22 developers find themselves in when we face a tight deadline without actually having any workable specs. CYA makes it tough to get sign-off on important specifications so you're told to "just do it" at the same time that you know you will be held responsible for any errors (perceived or actual) that eventually happen.

The second is that companies are in a struggle for survival so products and solutions literally cannot be completed soon enough. Whether it is first to market with a product or getting that internal app out to users desperately in need of them, you have time pressure that isn't under your control. You can't get away with telling people to chill and it'll be done when it is done because their need is now.

You asked for solutions, so I'll offer my paltry advice as well. If you are in a situation where that first concern is dominant, get out. It's an unhealthy environment and a company so busy looking inward for answers is just begging to be blind-sided by its competition--thus heightening the tension even further.

If you're in a situation dominated by the second, recognize that you are in the same situation every other programmer on the planet is and that how you deal with it can make a big difference all the way around. In dealing with it, I highly recommend learning to communicate well in terms a business can understand so that you can present options instead of merely listing objections. Asking for more time may truly be the right answer but it is more often helpful to present true alternatives to business leaders--alternatives based on data and dollars rather emotion, need, or even best practices. A business doesn't need to know all the latest advances in software development. TDD, Agile, BDD, all of that is uninteresting to them. They want alternatives expressed in terms of quality, time, and cost. In doing this, be prepared to compromise and be aware that it is their decision to make. Business leaders will sense and resent manipulation on your part, but that's a different topic...

Answer 12

I think the truth is in the middle of everything mentioned here in the answers before...

Personally (and from my daily experience) I think most problems come from incomplete specifications of customers (even though inhouse coworkers) and unaccepted (or not understood) timelines by project managers to satisfy customers.

Answer 13

Simply the fact that most company think you can switch a programmer for another without problem.

Answer 14

I think it's two things

1) The lack of time spent on requirements. Not just what is the app going to do but the testing plans and such.

2) The lack of time spent on testing. I think that testing is still seen as one of those things you can cut to crunch the timeline.

Answer 15

That there is a mismatch between academia and the commercial software industry.

Academia teaches and turns out software developers that prize exactly the wrong things and ignore exactly the right things. Software developers are trained to care deeply about a reliable process, correct code and predictable software. And they ignore popularity. But, to the industry, popularity (and the resultant sales) are the only thing that matters.

This is the major problem: if software developers value different things than their employers, everybody is relentlessly disappointed.

Answer 16

cool new stuff.

Just as we begin to write stable, maintainable systems.. someone comes along with a new language, framework, model etc and everone jumps on the bandwagon. And we get new bugs, design flaws and generally less relaible code. Of course, everyone has to jump on that bandwagon because interviewers suddenly want you with those skills, and you need to keep your career current.

Its no wonder there is so little quality code as its continually being thrown away and replaced with 'cool new stuff' that does mostly the same.

Answer 17

This is the difference between software development and software engineering.

To me, when I think of the problems I face developing software, I think of a lack of precision. There's too much "Hey, let's try this out and maybe it'll work."

Answer 18

3 issues:

The continued reliance and complete misguided belief in the waterfall approach to software development. It does not work, it never has, and it never will.

The lack of a continuous integration process, including automated builds within an organization.

Inadequate client acceptance testing throughout the entire life a software project.

Answer 19

Since Programming is kind of applied art, IMHO these are two main problems with software development:

1). Relying mostly on knowledge, not on experience and/or intuition.

2). Trying to convert Programming to engineering discipline.

Answer 20

My customers frequently ask me for something like "a new button that sends me a email" instead of asking me to solve problem underlying that request. There are many right ways and many wrong ways to build software, but customers do not trust Software Engineers to take care of it, like they trust engineers of other disciplines.

• Civil Engineers tell us how to build roads.
• Aerospace Engineers tell us how to fly.
• Chemical Engineers tell us how to create polymers.
• but Software Engineers are told how to build software.

I could create 99% awesome software if my customers respected what it takes to do the job right. The biggest problem with Software Engineering is that it is not venerated as an extremely difficult discipline, like other kinds of engineering. This leads to the kinds of Mythical Man Month conditions that we have to work under.

I think this might be because we programmers keep to our own, and we're happy to impress our clients with "yeah, we can do that" responses, since it makes us look capable and worth their money.

Answer 21

Not understanding that we don't know how long some things will take to produce as rarely is the initial design perfect. The idea here is that we can give a hypothetical solution but as that comes into focus some parts may not be so clear in terms of business logic elements, e.g. could you charge someone a tenth of a penny? That along with there rarely being sufficient structure so that there is forward movement in development where what is a priority is known: Is it maintainability, scalability, readability, execution time, execution space, ease of use, etc.

Answer 22

Lack of specs. Many projects start with "Can you do this?" "Yeah, sure."
Some time later: "Great. Can you add this: ...?" (It can be informal like this or more formal with requirements, schedules etc for larger projects.)
And so on. After a couple of iterations the code is ready to be thrown out and 100% re-written but nobody has the time/budget to do so. Not enough refactoring is done because there's pressure to get the next version out. And after a while the code is just horrible.

Answer 23

Software development is part engineering and part art. The tension between the engineering side and the art side is what makes software development exciting and at the same time difficult.

Answer 24

I'd say I'm in the Alan Kay camp. He is for Reinventing Programming and thinks programming is taught bad. To much code is doing same stuff.

Answer 25

Maintaining a top-quality coding team

Many of my coworkers are friendly, smart, experienced, reliable, and experts in their fields. I love working with these people. One or two are lacking - a very stubborn guy over here, maybe a slow worker over there. Somehow I feel much more influenced by these bad cases, who are often a bottleneck for productivity.

Answer 26

Daily tasks can get repetitive and boring after 1 or 2 years at the same place or in the same position. A company works on a specific product or product line and the tools and algorithms you can use are almost the same. It's always fun to work on a completely new feature.

Something similar is code maintenance. It's common that there is no technical specification or code comments. It can be pretty annoying to guess what was the intention of the coder. It gets more exciting when the code was written by a part-time worker/student. Unfortunately it's common here to leave a company after 1-2 years.

Answer 27

Have you read or know about David Platt's Why Software Sucks? He has written a book.

Answer 28

I think the problem can be sumarized with an (old) joke:

In the Beginning was The Plan

And then came The Assumptions

and The Assumptions were without Form

And The Plan was completely without Substance.

And The Darkness was upon The Face of The Workers.

And they spoke amongst themselves, saying

"It is a Crock of Shit, and it stinketh."

And The Workers went unto their Supervisors and sayeth,

"It is a Pail of Dung and none may abide in the odor thereof."

And The Supervisors went unto their Managers and sayeth unto them

"It is a Container of Excrement and it is very strong,

such that none may abide by it."

And The Managers went unto their Directors and sayeth,

"It is a Vessel of Fertilizer, and none may abide its strength."

And The Directors spoke amongst themselves, saying one to another,

"It contains that which aids Plant Growth, and it is very strong."

And The Directors went unto The Vice Presidents and sayeth unto them,

"It promotes growth and is very Powerful."

And The Vice Presidents went unto The President and sayeth unto him,

"This New Plan will Actively promote the growth and efficiency

of this Company, and these areas in particular."

And The President looked upon The Plan

and saw that it was good, and The Plan became Policy.

And this is how "SHIT HAPPENS!"

Answer 29

There are really two problems: bad programmers and bad project managers.

And bad customers of course.

Answer 30

Money. Most of the issues with shoddy project management, unrealistic deadlines, and bad programmers (cheap programmers hired for less i.e. offshore devs) have to do with money. What's interesting is that software in many cases is built to save money, but often the customer suffers from sticker shock and doesn't want to pay like they used to. As a result of this, sales and project management start to cut quality by decreasing the requirements gathering time, and customer cost expectations so as not to lose the deal.

Also I believe that many non-software people are starting to view software with less value and respect than before. They tend to believe that it's really easy to just slap some stuff together, and it should be simple and easy (i.e. can't you just do it Share Point in a day attitude). Because of this they feel they shouldn't be paying as much as they did in the past for similar offerings.

In the end the developers are left holding the bag, and have to bear the brunt of pressure to churn out the impossible within an unrealistic time frames and budget. The sad thing is, I already see this problem getting worse with this bad economy. I hope that something improves and that our field is not turned into commodity, where we are devalued and our earning power is diminished. I sure don't want to do this for $20.00/hr!

Answer 31

Legacy code.

It's the reason that so much of desktop development continues to involve C and C++ (or at least involves coding to C APIs) even when easier and more modern languages (everywhere from C# to D to Python) are available: legacy apps must be maintained, and even new libraries and APIs must support the lowest common denominator.

It's the reason that all major operating systems have to support APIs dating back 20 or 30 years (Windows 1.0 and POSIX).

Users' need for maintain compatibility with legacy code and legacy data makes software prone to vendor lock-in, which stifles competitiveness and creativity.

Developers' need to maintain compatibility with legacy code can take tremendous effort, as Joel Spolsky and Raymond Chen write about.

The need to remain compatible with legacy code is much of the reason that Vista took so long; failure to remain compatible is much of the reason that Vista did so poorly.

Legacy web browsers are the reason that web development has to cater to DOM incompatibilities, CSS hacks, etc. Legacy web sites are the reason that web browsers can't more easily advance. The legacy of the whole web ecosystem keeps us using technologies that originated in the Netscape era instead of easily switching to something completely different (like GWT or Silverlight).

The need to maintain bad legacy code ties up enormous programming resources. ("One bad programmer can easily create two new jobs a year.")

Answer 32

Mediocre requirements implemented by mediocre developers and architects under truncated time lines with an emphasis on getting it done fast rather than well.

Answer 33

Also, I would highly recommend listening to this podcast episode on DNR (dotnetrocks).

very insightful discussion: Pat Hynds on why projects fail