Risk evaluation for framework selection

Question

I'm planning on starting a new project, and am evaluating various web frameworks. There is one that I'm seriously considering, but I worry about its lasting power.

When choosing a web framework, what should I look for when deciding what to go with?

Here's what I have noticed with the framework I'm looking at:

• Small community. There are only a few messages on the users list each day
• No news on the "news" page since the previous release, over 6 months ago
• No svn commits in the last 30 days
• Good documentation, but wiki not updated since previous release
• Most recent release still not in a maven repository

It is not the officially sanctioned JEE framework, but I've seen several people mention it as a good solution in answers to various questions on Stack Overflow.

I'm not going to say which framework I'm looking at, because I don't want this to get into a framework war. I want to know what other aspects of the project I should look at in my evaluation of risk. This should apply to other areas besides just JEE web, like ORM, etc.

Answer 1

All the signs you've cited could be bad news for your framework choice.

Another thing that I look for are books available at Amazon and such. If there's good documentation available, it means that authors believe it has traction and you'll be able to find users that know it.

The only saving grace I can think of is relative maturity. If the framework or open source component is mature, there's a chance that it does the job as written and doesn't require further extension.

There should still be a bug tracker with some evidence of activity, because no software is without bugs (except for mine). But it need not be a gusher of requests in that case.

Answer 2

Here are the things I look for in a framework before I decide to use it for a production environment project:

• Plenty of well laid out and written documentation. Bad documentation just means I'm wasting time trying to find how everything works. This is OK if I am playing around with some cool new micro framework or something else, but not when it's for a client.

• A decently sized community so that you can ask questions, etc. A fun and active IRC channel is a big plus.

• Constant iteration of the product. Are bugs being closed or opened on a daily/weekly basis? Probably a good sign.

• I can go through the code of the framework and understand what's going on. Good framework code means that the projects longterm life has a better chance of success.

• I enjoy working with it. If I play with it for a few hours and it's the worst time of my life, I sure as hell won't be using it for a client.

I can go on, but those are some primary ones off the top of my head.

Answer 3

Besides looking at the framework, you also need to consider a lot of things about yourself (and any other team members) when evaluating the risks:

• If the framework is a new, immature, "bleeding-edge" framework, are you going to be willing and able to debug it and fix or work around whatever problems you encounter?

• If there is a small community, you'll have to do a lot of this debugging and diagnosis yourself. Will you have time to do that and still meet whatever deadlines you may have?

• Have you looked at the framework yourself to determine how good it is, or are you willing to rely on what others say about it? Why do you trust their judgment?

• Why do you want to use this rather than the "officially sanctioned JEE framework"? Is it a pragmatic reason, or just a desire to try something new?

• If problems with the framework cause you to miss deadlines or deliver a poor product, how will you talk about it with your customer?

Answer 4

I'll say that so-called "dead" projects are not that great a danger as long as the project itself is solid and you like it. The thing is that if the library or framework already does everything you can think you want, then it's not such a big deal. If you get a stable project up and running then you should be done thinking about the framework (done!) and focus only on your webapp. You shouldn't be required to update the framework itself with the latest release every month.

Personally, I think the most important point is that you find one that is intuitive to your project. What makes the most sense? MVC? Should each element in the URL be a separate object? How would interactivity (AJAX) work? It makes no sense to pick something just because it's an "industry standard" or because it's used by a lot of big-name sites. Maybe they chose it for needs entirely different from yours. Read the tutorials for each framework and be critical. If it doesn't gel with your way of thinking, or you have seen it done more elegantly, then move on. What you are considering here is the design and good design is tantamount for staying flexible and scalable. There's hundreds of web frameworks out there, old and new, in every language. You're bound to find half a dozen that works just the way you want to think in your project.

Points I consider mandatory:

• Extensible through plug-ins: check if there's already plug-ins for various middleware tasks such as memcache, gzip, OpenID, AJAX goodness, etc.
• Simplicity and modularity: the more complex, the steeper the learning curve and the less you can trust its stability; the more "locked" to specific technologies, the higher the chances that you'll end up with a chain around your ankle.
  • Database agnostic: can you use sqlite3 for development and then switch to your production DB by changing a single line of code or configuration?
  • Platform agnostic: can you run it on Apache, lighttpd, etc.? Could you port it to run in a cloud?
  • Template agnostic: can you switch out the template system? Let's say you hire dedicated designers and they really want to go with something else.
• Documentation: I am not that strict if it's open-source, but there would need to be enough official documentation to enable me to fully understand how to write my own plug-ins, for example. Also look to see if there's source code of working sites using the same framework.
• License and source code: do you have access to the source code and are you allowed to modify it? Consider if you can use it commercially! (Even if you have no current plans to do that currently.)

All in all: flexibility. If I am satisfied with all four points, I'm pretty much done. Notice how I didn't have anything about "deadness" in there? If the core design is good and there's easily installable plug-ins for doing every web-dev 3.0-beta buzzword thing you want to do, then I don't care if the last SVN commit was in 2006.