Concrete symptoms of over-engineering

Question

I have recently found myself in the position of explaining an (In-House) application I have written to two candidates my company likes to hire in order to assist in maintenance and adding minor features.

It is the first "production" application I have written, it has 45k LOCs and I spent almost two years of "solo" development on it. I am fairly young (18) and wrote the application from scratch while being contracted as stand-in for a former developer who left the company. Unexperienced in designing applications of this size, I tried to use common architecture- and design-patterns.

Today I know I have done some serious over-engineering, e.g. using a disconnected change tracking architecture instead of the Unit Of Work pattern, which the chosen ORM has already implemented. I will probably never have to go "real" three tiers.

Both candidates have 10 years+ background in In-House Application Development with the relevant platform. Being half their age and having little experience I do respect their opinion. When I was explaining the application architecture to them, comments were along the lines of:

• Jeez, no one would pay me to do stuff like that, I have to get things done
• Stick with what the framework does, don't use fancy libraries/technologies
• Don't wrap framework code. On a team, everyone will write his own wrapper code anyway.
• You're using .NET 3.5? Well, we are using 2.0.
• What does that LINQ stuff buy me? All this query composition and projection seems too complicated.

Now I am asking myself:
Am I an architecture astronaut? How do I know I am going too far with architecture? What are common symptoms of over-engineering?

Answer 1

Avoiding any use of YAGNI, DRY, and KISS come to mind in looking at things that are over-engineered. If there are many parts that seem to be partially completed and many parts of code that seem to have a, "What if this happens? What if that happens?" feel to it, that would be another point. Ignoring good principles of OO design or SOLID principles would be another to note. If you think you have written the perfect code, that would be another sign of trouble as it is extremely rare for anyone to write something that can't be improved in one way or another.

IMO, beware that some people may be overly critical of your work as part of any code base can involve people liking things a certain way, e.g. naming conventions on methods, tests and variables. That's just the way it is. Now, what you may have to figure out is how to handle people in siutations such as conflict or persuasion/influence, where there are tools that can help.

Answer 2

As for the question about if you an architecture astronaut: If you are aware of the danger that puts you ahead of a lot of people. You don't want to go the way of your cow-orkers either, it sounds like some of them have become crusty old whiners.

Over-engineering is the result of a problem with prioritization that resulted in some part of the system getting too much attention. So the most apparent symptom of over-engineering would be that you can see all around other parts of the system that are hurting for lack of attention.

(There is also a tendency for over-engineering to expose the system to increased risks of bad design, because of increased complication and the amount of error-prone speculation involved in deciding what aspects to over-engineer, but as a comment points out, that doesn't automatically follow.)

Answer 3

What are common symptoms of over-engineering?

Code that solves problems you don't have.

Answer 4

Did you implement your project within a reasonable time frame? Is it working right now? If so, you can probably relax a little bit, as one of the worst problems with over engineering is never actually getting anything useful done.

The guys you talked too may have some good suggestions, but it doesn't mean you need to take everything they say as gospel. For example, using a recent version of .NET on a fresh project does not strike me as anything to worry about. Are they really complaining about that?

Answer 5

For most in-house business applications, most of your code should be concerned with implementing business concerns, and not technical concerns unrelated to the business (like your "disconnected change tracking architecture"). The currently available frameworks are pretty mature and support most common use cases. If you're inventing new technology or (in the context of business-application development) just wrapping some other existing framework or library just for the sake of wrapping, you're probably doing it wrong. Ideally every piece of architecture you build should be traceable back to some business requirement. Keep it simple.

Answer 6

IMHO most of the comments you got about your application aren't really about over-engineering because over-engineering is not about technology. It's about architecture. New technologies can be learned and understood in a reasonable amount of time. Understanding an over-engineered application is usually much more difficult and sometimes even impossible. This makes the points 2, 4 and 5 invalid. The first point is not really valid because you obviously got paid for writing the application as it is and if it works you got no problem here.

This is my "quick test" to find out if an application tends to be over-engineered:

• Wrappers for "everything": Wrappers are useful but it's easy overdoing it. Check if you only wrap things that really need to be wrapped. (I basically wrapped my own wrapper once. I know what I'm talking about ;-) .)
• Reinventing the wheel: A classic. This is very common and you already mentioned it. Did you implement some functionality because you needed to your wanted to? What does your framework do what other available libraries don't?
• "Feels" over-engineered: This is the most important point but also the hardest point to see. Take a look at your code and look which parts feel overly complicated. Ask your self then if there is a easier way to implement it and why you didn't choose this way. If you got no good answer this part is probably over-engineered.

These are just quick tips which I use for my applications. They are not guaranteed to be the be-all and end-all of "over-engineering detection".

Answer 7

Boredom

Boredom is good precursor to over-engineered code. I'll admit, when I got my first job, I felt so underutilized. I was just bored. And when I got bored, I wrote code. Not just any code -- CATHEDRALS OF CODE.

No seriously, I had a mental picture of my code and abstractions as large towers with golden jutting spires, flying buttresses of glassy onyx, a wonderful vault supporting by arched domes topped with beautiful geometrical tracery, etc etc etc.

It was really fascinating to see the patterns working together for myself, but in retrospect, I am completely ashamed of the ungodly mess I left behind.

If you're writing your own frameworks and DSLs code to while away the less stimulating hours at work, just stop. Time is better spent reading Wards Wiki, or writing an open source book, or you may just want to ask management for more work.

Answer 8

You're not an architecture astronaut. LINQ is pretty simple and basic and useful, for one. Same goes for .NET 3.5.

At the same time, you're the newbie on the team and going to get some kind of ribbing, even if they like what you did.

Take it all with a grain of salt. Just accept their criticism, nod, and have a beer with them afterwards.

If they ask you to change it, then your comment is "jeez .. I know i did it wrong, but it works and it's going to be too much trouble to change".

Answer 9

Writing your own Framework

Odds are, someone's already done it. More than that, they've already done it 1000x better than you ever could. More than that, whatever they've done is probably already an industry standard, so that learning the technology will make you more competitive at other jobs.

At the last company where I worked, a programmer had worked solo on his projects for most of his tenure. He wrote one of the company's more popular apps and was widely regarded as the best on the team -- but in my opinion, he had a nasty habit of writing everything he needed from scratch.

He'd written his own dependency injection framework, his own ORM, a unit testing framework (which, inexplicably, looked and acted very similar to NUnit -- why didn't he used NUnit?), a framework for creating factory objects (a "factory factory" I'd call it).

Mind you, the code was actually remarkable, but what was the point?

Writing a better Core Library

At my current company, it always seemed like programmers writing useless amounts of code to replicate features already present in the .NET framework.

Among other things, they wrote:

• An active-directory framework for forms authentication in ASP.NET webforms -- inexplicable because ASP.NET has this function built-in.
• Hot-swappable themes and skins for websites -- also inexplicable, since the code was less functional than built-in ASP.NET themes and required 1000% more bloat.
• They inexplicably wrote their own typed data sets and data adapters. These objects provided less functionality than typed datasets which VS will autogenerate for you, while simultaneously requiring more boilerplate code than NHibernate domain objects.

Either they don't know the framework very well, or they think its notoriously inadequate.

There are only preciously few examples I can think of where the re-implemented library is better than the original (see Jane Street Core Library, C5 Generic Collections for .NET, a real currency class), but odds are, you won't write a better standard library.

Answer 10

half of it is old men (i'm in their league) sticking to the tried and true steam engine they've known. the other half is true, but not really telling you anything new.

other than that, Jeff Sternal's answer is stellar.

Answer 11

One very strong warning sign of overengineering is when everything goes through so much indirection that it's hard to find the piece of code that actually implements some concrete, domain-level piece of functionality. If you find that most of your functions do very little concrete work and just call other virtual functions, you may have a problem.

Answer 12

Plugins which provide intrinsic functionality to your app

Let's face it, pluggable architectures are just damn sexy and fun to write. However, this is another one of those areas where you need to ask yourself, "do I really need to do it this way?".

If you don't have a need for ad-hoc additions to your application, don't expect anyone to write third-party extensions, and the scope of your application fairly well-defined, you don't need a pluggable architecture.

You shouldn't write plugins to support intrinsic functionality in your app. Let's say you were writing a Paint program; you'd probably support plugins to save files in multiple formats, but you wouldn't need a pluggable undo manager or file browse dialog.

Answer 13

When a co-worker’s computer comes flying past your head because they’ve spent the last 6 hours trying and failing to make a freaking dialog box appear using your ridiculous framework, that’s a pretty concrete symptom right there.

Answer 14

Try to evaluate if you have done things to minimize work over the expected life of the code. This includes maintenance as well as development.

In the code lifecycle, remember that the life of the code is not the same as the life of the application. It maybe better to write a quick prototype first, then re-engineer/refactor after better understanding of the domain. In this situation, the lifecycle is very short, so keep it simple.

In addition, different applications require different amouhts of engineering. Is this application going to cost lives if it fails? I.e. is it a controller for a mechanical heart, or for the space shuttle navigation rockets? Or is it a contact list for bored teens?