What do you feel is over-generalization?

Question

Having spent some time playing around in Haskell and other functional languages, I've come to appreciate the simplicity of design that comes from describing problems in general terms. While many aspects of template programming can be far from simple, some uses are common enough that I don't think they're an impediment to clarity (especially function templates). I find templates can often simplify the current design while automatically adding a bit of future-resistance. Why should their functionality be relegated to the library writers?

On the other hand, some people seem to avoid templates like the plague. I could understand this a decade ago when the very concept of generic types was foreign to much of the programming community. But now all of the popular statically-typed OO languages support generics of one form or another. The added familiarity seems to warrant an adjustment of the conservative attitudes.

One such conservative attitude was expressed to me recently:

You should never make anything more general than necessary - basic rule of software development.

I was quite honestly surprised to see this stated so dismissively as if it should've been self evident. Personally I find it far from self-evident, what with languages like Haskell where everything is generic unless you specify otherwise. That being said, I think I understand where this point of view comes from.

In the back of my mind, I do have something like that rule rattling around. Now that it's at the forefront, I realize I've always interpreted it in the light of overall architecture. For example, if you have a class, you don't want to load it up with tons of features you might one day use. Don't bother making interfaces if you only need one concrete version (though mockability might be a counterargument to this one). Things like that...

What I don't do, however, is apply this principle on the micro level. If I have a small utility function that has no reason to be dependent on any particular type, I'll make a template.

So what do you think, SO? What do you consider to be over-generalizing? Does this rule have differing applicability depending on the context? Do you even agree this is a rule?

Answer 1

Something is over-generalized when you're wasting time generalizing it. If you are going to use the generic features in the future then you're probably not wasting time. It's really that simple [in my mind].

One thing to note is that making your software generalized isn't necessarily an improvement if it also makes it more confusing. There is often a trade off.

Answer 2

Over generalizing makes me crazy. I'm not scared of templates (nowhere near) and I like general solutions. But I also like solving the problem for which the client is paying. If it's a one week project, why am I now funding a one month extravaganza which will continue to work not only through obvious possible future changes like new taxes, but probably through the discovery of new moons or life on mars?

Bringing this back to templates, the client asks for some capability that involves your writing a function that takes a string and a number. You give me a templated solution that takes any two types and does the right thing for my specific case and something that might or might not be right (due the absence of requirements) in the rest of the cases, and I will not be grateful. I will be ticked off that in addition to paying you I have to pay someone to test it, someone to document it, and someone to work within your constraints in the future if a more general case should happen to come along.

Of course, not all generalization is over generalization. Everything should be as simple as possible, but no simpler. As general as necessary, but no more general. As tested as we can afford, but no more tested. Etc. Also, "predict what might change and encapsulate it." All these rules are simple, but not easy. That's why wisdom matters in developers and those who manage them.

Answer 3

If you can do it in the same time, and the code is at least as clear, generalization is always better than specialization.

There's a principle that XP people follow called YAGNI - You Ain't Gonna Need It.

The wiki has this to say:

Even if you're totally, totally, totally sure that you'll need a feature later on, don't implement it now. Usually, it'll turn out either a) you don't need it after all, or b) what you actually need is quite different from what you foresaw needing earlier.

This doesn't mean you should avoid building flexibility into your code. It means you shouldn't overengineer something based on what you think you might need later on.

Answer 4

there are 2 examples from microsoft in over-generalization:
1.) CObject (MFC)
2.) Object (.Net)

both of them are used to "realise" generics in c++ which most of the people doesn't utilize. In fact, everyone did type checking on parameter given using these (CObject/Object) ~

Answer 5

I think you should consider two basic principles of programming: KISS (keep it simple and straightforward) and DRY (don't repeat yourself). Most of the time I start with the first: implement the needed functionality in the most straightforward and simple way. Quite often it's enough, because it can already satisfy my requirements. In this case it remains simple (and not generic).

When the second (or max third) time I need something similar I try to generalize the problem (function, class, design, whatever) based on the concrete real life examples -> it's unlikely that I do the generalization just for itself. Next similar problem: if it fits to the current picture elegantly, fine, I can solve it easily. If not, I check if the current solution can be further generalized (without making it too complicated/not so elegant).

I think you should do something similar even if you know in advance that you will need a general solution: take some concrete examples, and do the generalization based on them. Otherwise it's too easy to run into dead ends where you have a "nice", general solution, but it's not usable to solve the real problems.

However, there might be some exceptional cases to this.
a) When a general solution is almost exactly the same effort and complexity. Example: writing a Queue implementation using generics is not much more complicated then doing the same just for Strings.
b) If it's easier to solve the problem in the general way, and also the solution is easier to understand. It does not happen too often, I can't come up with a simple, real life example of this at the moment :-(. But even in this case having/analyzing concrete examples previously is a must IMO, as only it can confirm that you are on the right track.

One can say experience can overcome the prerequisite of having concrete problems, but I think in this case experience means that you have already seen and thought about concrete, similar problems and solutions.

If you have some time you could have a look at Structure and Interpretation of Computer Programs. It has a lot of interesting stuff about how to find the right balance between genericity and complexity, and how to keep the complexity at a minimum that is really required by the your problem.

And of course, the various agile processes also recommend something similar: start with the simple, refactor when it's needed.

Answer 6

Too generic ? I must admit I am a fan of Generic Programming (as a principle) and I really like the idea that Haskell and Go are using there.

While programming in C++ however, you are offered two ways to achieve similar goals:

• Generic Programming: by the way of templates, even though there are issues with compilation time, dependency on the implementation, etc..
• Object-Oriented Programming: its ancestor in a way, which places the issue on the object itself (class/struct) rather than on the function...

Now, when to use ? It's a difficult question for sure. Most of the times it's not much more than a gut feeling, and I've certainly seen abuse of either.

From experience I would say that the smaller a function/class, the more basic its goal, the easier it is to generalize. As an example, I carry around a toolbox in most of my pet projects and at work. Most of the functions / classes there are generic... a bit like Boost in a way ;)

// No container implements this, it's easy... but better write it only once!
template <class Container, class Pred>
void erase_if(Container& c, Pred p)
{
  c.erase(std::remove_if(c.begin(), c.end(), p), c.end());
}

// Same as STL algo, but with precondition validation in debug mode
template <class Container, class Iterator = typename Container::iterator>
Iterator lower_bound(Container& c, typename Container::value_type const& v)
{
  ASSERT(is_sorted(c));
  return std::lower_bound(c.begin(), c.end(), v);
}

On the other hand, the closer you get to the business specific job, the least likely you are to be generic.

That's why I myself appreciate the principle of least effort. When I am thinking of a class or method, I first take a step backward and think a bit:

• Would it make sense for it to be more generic ?
• What would be the cost ?

Depending on the anwsers, I adapt the degree of genericity, and I struggle to avoid premature locking, ie I avoid using a non-generic enough way when it doesn't cost much to use a slightly more generic one.

Example:

void Foo::print() { std::cout << /* some stuff */ << '\n'; }

// VS

std::ostream& operator<<(std::ostream& out, Foo const& foo)
{
  return out << /* some stuff */ << '\n';
}

Not only is it more generic (I can specify where to output), it's also more idiomatic.