Is Software Engineering Dead?

Answer 1

My view is that software has never been on a par with engineering. It lacks the scientific basis that physics gives to engineering.

Civil engineering has been with us since the pyramids and aquaducts; mechanical, chemical, and electrical engineering have been making rapid progress since the late 19th century. Software has only been around since the 50s. It hasn't had the same time to develop that engineering has.

Software development has more of the feel of writing or art or craft than engineering does. All the UML and false metrics like KLOC and code coverage won't make it so. I think the analogies with building ("architecture") and manufacturing ("widgets", "metrics") don't do it justice.

That isn't to say that software development isn't a wonderful activity with its own rigors and challenges. It's just not "engineering" yet and may never be. Tom DeMarco appears to have lost faith.

UPDATE: The fact that 27 years have passed since DeMarco's metric book came out, and we still don't have an agreed-upon way to measure quality or productivity or value in software says that it hasn't delivered on its initial promise.

I always laugh when I hear about applying Six Sigma techniques to software. The people who suggest that have never been on a shop floor, and have little idea of how a tolerance band for a given dimension figures into its success in that domain.

When I was practicing as a mechanical engineer I could count the number of people with "engineer" in their job description without a bachelor's degree in the field from an accredited institution on one hand. While I'd agree that a degree is no guarantee of quality or above average results for all, it's an important difference that separates it from computer science and software development. The barrier to entry in programming and software development is a lot lower than it is for professions like engineering, law, or medicine. Every person who wrote Basic on an early PC can be considered a programmer. Not so with other fields.

Answer 2

This sounds alot like the software, art or engineering? arguments that go around.

IMHO, software engineering assumes fixed constructs like duffymo stated. Mechanical engineering has the laws of physics to contend with.

In programming we have other rules such as business constraints, chip constraints and general constraints of computation and complexity theory that give foundation to the topic as an engineering discipline.

Computer software is engineering when you work within fixed bounds, often applied by yourself. Computing is so abstract and fluid that software engineering is almost as odd as "mathematical engineering" ... Engineering of what?

I think that software is engineered within arbitrary bounds, implied by business and complexity costs. Computing, and in turn software however is not fixed to these bounds, allowing its concepts to become an art without boundary other than those artificially or physically imposed.

So long as software must be created within some fixed boundary of cost, time, complexity or the other, it will be software that is engineered.

When those bounds don't exist, it is like mechanical engineering, without the constraints of physics.....

An amazing blank canvas for art.

But I am no expert.

addition:

In software, the same people design and then build. How many mechanical engineers or architects are plastering, welding and riveting their designs?

After reading all the other comments, I have come to a personal conclusion that from software engineering through to electrical or mechanical engineering ... both design processes involve some form of science along with current and past methodologies. These methodologies are used to design a system, of any kind, within some design constraints.

Doesn't really matter if this is a car, bridge or otherwise. You have convention and innovation that breaks convention. The design is the art. Keeping it within the bounds of the brief is a composite of design and engineering the art-form into constraints.

Then comes the execution, building or implementation, both mechanical, electrical, software or other have the same problems; managing expectations of the construction of a given design. I am sure bridges, buildings, car production and circuit productions run over their budgets and time constraints just like software.

In the end, IMHO, metrics are applied to the management of implementation and the requirements of design, regardless of the underlying science or design discipline. The design itself is an art. So I would consider mechanical design an art, just as is architecture or software design.

What makes things different is that software designers are often the builders too. Im not sure too many architects/mech.engs get are also welding the frameworks of their buildings. In this way, software has a bias because as software designers and builders our hands are involved in both sides ... but they must be treated in a separate regard when looked at for measurement of any kind.

In my very humble opinion, software is no different from other engineering disciplines I think, but our separation of design and implementation in terms of measurement is not the same as other professions, and the boundaries blur.

Answer 3

I admit beforehand that my answer is an over-simplification. That said, I think there is an essence of truth in what I say below.

In regular engineering there is discipline that is applied almost universally. Rules and standards that are enforced. Specify an non-trivial engineering problem and it is possible to construct a methodology for solving that problem that broad numbers of engineers would agree upon. Follow that methodology and success is likely, even though you will still face problems and setbacks. Deviate from the methodology and success is elusive and the project may fail.

I have worked as a programmer at a lot of places, some very big and some very small. Rules varied from company to company, and in some cases from person to person. Some places enforced their rules, most did not. It worked upon the honor system. Specify a non-trivial computer software problem and there may be no consensus for how to proceed. Regardless of the methodologies chosen, there is an equal chance the project may fail.

Like it or not, computer programming has more in common with art than with engineering. Creating software has become a curious blend of mathematical theorem proving and non-fiction writing. Which is what I found attractive about it in the first place.

Answer 4

I feel that programming and software design is analogous to apprenticed crafts of old, more like blacksmithing than accounting. It can be learned, and perhaps mentored, but never taught. Anyone who's ever worked with someone whose sole qualification as a programmer is a Computer Science degree might concur. The principles of software design are vague, and often contradictory. To make matters worse, their over-application can be as disastrous as their under-application. There are guidelines, but no simple rules.

The fact that we separate the good from the bad my "smell" indicates to me that the endeavor is intuitive in nature. It is art, and one of the primary goals is beauty. Try telling this is your boss, who is more interested in talking about deliverables. The notion that something that the primary users of the product will never see, that only a small percentage of the population has any change of understanding (let alone appreciating) -- the notion that this thing must be beautiful sounds absurd. But we all know: it must be.

It's one of the few disciplines in which several people (or perhaps many more than that) are expected to collaborate on a work of art for profit. To my knowledge, even architecture doesn't attempt this to the same degree. I've always thought that architecture is one of the best analogies we have, but I'm not sure how to utilize that analogy. Can we take inspiration from their organization structure or design process? Or is our craft just different enough that we're on our own? They manage to control their output and deliver on time -- what's our malfunction, as an industry?

The question isn't really whether the production of software is different from just about every other process on earth, but exactly why that is, and what we should do about it.

Answer 5

Whenever this 'engineering or not' debate comes up, I am amazed by peoples understanding of engineering. 'Strict rules that are always followed', 'discipline that is applied almost universally', 'exact numeric answers', etc, etc. For SW development OTOH you read 'There are guidelines, but no simple rules' and similar statements.

When I started university, I believed those, too.

After several years working with process and mechinical engineers, I know differently. They are working just as chaotic and cluless as any software developer.

---

Edit:

Quoting from areply to Jeff's blog entry: 'In civil engineering, and most other engineering disciplines, there is only a 'correct' way. Either you design the bridge to support the correct amount of weight or it falls down.'

Wrong. Wrong, wrong, wrong. Wrong!

There are at least as many ways to design a bridge as there are ways to design a string class. If you design a bridge that not only supports the required weigth, but tenfold the required weight, you did it wrong, because you could have lowered the cost by using less material and completing it faster (less wages to be paied) etc.

If you design a new gear for a car, that needs less maintenance, but requires the whole motor to be removed from the car in order to replace the old gear, you did it wrong, because in complex systems, there are always more aspects to consider than just your module, you also need to think of it's interaction with the rest. That should sound familiar to a developer, shouldn't it?

Answer 6

I shure hope software engineering is not dead... I'am presently in my 3rd year as a Software engineering student! The fact that universities actually make the distinction bwtween computer science, computer engineer and software engineer tells us something, doesn"t it?

For what you guys say about software not beeing "scientific" enough, you need to come up to date on the subject!

I don't think software engineering is going to go very far, especially since projects are become bigger and more complicated by the second!

Answer 7

I hadn't read DeMarco's original work. I now know who to blame for managers who think only in terms of metrics.

From early in my career, I felt it important to make a distinction between people who just write code and people who are "professional" about it. That is, people who understand that process is necessary to some extent, understand the concept of source control, etc. I didn't know what to call those people until I heard the term "Software Engineer".

If I'd known I was using a term that implied that I thought I could control things by measuring them, I'd have stuck with "computer programmer".

I like the term "Software Craftsman", but it doesn't leave me with a good feeling about whether such a person would work well in a team. It seems to me that you can be a craftsman, but only check in your code once a week, or not get your code done in time for QA to test it. I'll be on the lookout for another term to use.

Answer 8

Agree with most of what's been said here, but I just wanted to pick apart the sound-byte, and it is pure hyperbole sound-byte:

"control is ultimately illusory on software development projects"

Control is illusory in all aspects of everything to a greater or lesser degree. That's just a consequence of quantum physics, entropy, or humans being humans depending on the scale of your subject. Managing software development has never been about control (at least not for a good manager), it's about minimising risk and compensating for failures. Anyone who says otherwise is selling something.

Answer 9

I don't consider software engineering dead because I think there are enough people out there (mostly engineers) who would like the software development process to be more like engineering, who will keep trying to make it more like engineering, and who will keep trying to convince people (especially programmers) that it should be more like engineering.

Personally, I don't think engineering principles work too well for software in general, but software engineering's apparent lack of success thus far will only be taken as evidence by engineering evangelists that there must be greater effort in advancing their cause.

Answer 10

Software engineering never was an engineering to begin with. Engineering is a hard science with cumulative and cooperative progress.

I can't sum it up much better than Richard Hamming:

Indeed, one of my major complaints about the computer field is that whereas Newton could say, "If I have seen a little farther than others, it is because I have stood on the shoulders of giants," I am forced to say, "Today we stand on each other's feet." Perhaps the central problem we face in all of computer science is how we are to get to the situation where we build on top of the work of others rather than redoing so much of it in a trivially different way. Science is supposed to be cumulative, not almost endless duplication of the same kind of things.

Answer 11

The basic flaw in the DeMarco approach is that it was never software engineering, it was software production engineering. He accidentally slipped a level of meta in there.

Software metrics are executable file size, memory usage, latency, ... Software production metrics are lines of code, defect count, complexity, ...

Software production metric possibly matter in the case where you need to manually produce 10,000 or more similar software systems: that's about the scale where hardware production engineering kicks in as worthwhile. You don't build a factory and establish an optimised production line to build a few hundred cars a year. And very few software firms are on that scale: even IBM produces far fewer software products per year than Rolls Royce produces cars.

You might find two or three organisation worldwide that would benefit from a robust set of software production engineering techniques. Places like the US army, NASA and the NHS.

But there are far more who would benefit from an appropriate level of actual software engineering skills, metrics and techniques.

Answer 12

When I was at university, I was told other engineering disciplines were much better at engineering. I believed my professors.

What I discovered, working with Electronic and Mechanical engineers over 15 years, is that other disciplines mostly don't do something nobody's ever done before. If you asked a Civil engineer to do something nobody has ever done before: their reaction would be to tell you it was going to cost enormous amounts of money and would have an uncertain schedule. And nobody would tell a civil engineer what kind of tools or parts to use.

Bridges, Engines, Cars, Telescopes....

As things get more and more complicated, and more and more novel, "traditional" engineering disciplines start looking more like software engineering. One place I worked invented the dishwasher that looks like a filing cabinet. The mechanical design had bugs just like software. They fixed it eventually ( well enough to sell.)

The contradiction is that beyond simple things, the engineering approach to most things these days is to have software in the middle making it all work.

A car company ran an advertisement when I was a youngster about spending 6 million developing their new car. (Toyota Camry IIRC) Toyota had made a car that was different to their other cars. It cost them millions.

I think that if you measure the complexity of an electronic circuit, or mechanical system it is not anywhere near the complexity of "simple" software. Complexity in the mathematical state space/chaos theory sense of the word.

Consider a standard suspension bridge: no moving parts. Some pivots, some vibration dampers. Copy the bridge for your next project.

A software equivalent would be what ? We don't (on the whole) do critical and simple in software.

Lets look at manufacturing:

Software: Design software, Write program, hit compile. Copy onto a CD. The manufacturing bit was at the end. The copying of the data.

Mechanical Engineering: Design system: design assembly, design parts. Repeat for every one you want (Get parts made, assemble parts. Test, align)

The manufacturing bit was on another line and had to be repeated for every instance.

WARNING: Car Analogy.

We design a new thing every time we write software. Every program is not an Aston Martin, but a custom built car. Some will be from Monster Garage and some will be supercars from a guy in Italy with price tags in the tens of millions.

The funny part it that we can duplicate the "supercar from Italy" in 3 seconds and sell it to everyone in the world. So the cost drops to "shrinkwrap" software prices.

Software is different to other disciplines : because it is uncommon to design things that are not software from scratch.

Software not designed from scratch is really common. But still very complex. So, unreliable. After all, we're only selling it for $20.

END Car Analogy.

If a company said that the CEO was getting a car designed for him that the company was going to pay for, the shareholders would geek.

If the same company gets software written to their "business processes", nobody blinks.

(I lied about the car analogy ending :-) )

Answer 13

DeMarco's article is a reflection of current trends seen all around the development community lately, I think. People are exploring the potential space when it comes to software development models, possibly because the software engineering aspect is currently a bit tired.

Engineering is just one aspect of creating good software. It's the "do it again" aspect -- learn what you can from your past experiences, use that knowledge to measure your current performance, and control your future results. But the engineering process is built to reproduce, not to create. The engineer trains his entire career to be able to take a model that works and apply it with precision as the core component in something new.

But, as DeMarco points out, new ideas, the stuff that's really significant, is about creating, not reproducing.

Personally I've always found that the science model works much better for developing new software -- a mindset of discovery and testing. Perhaps software management should try to model itself more closely to science management than engineering.

But regardless, the lessons learned from the software engineering movement aren't about to be squandered or forgotten -- I continue to believe there is real value in applying the correct amount of measurement and control to software practices, it's just a matter of recognizing what parts of a project can be tightly controlled, and which cannot.