What are best practices for extracting a framework from an application?

Question

I've been working on an application for a little over two years and have developed a lot of useful helpers, utilities, features, setup conventions, etc.

I want to start building a similar application and think I could reuse a lot of the functionality built into the existing application with the new application.

I've always heard is it best to extract a framework from an application but not heard much about best practices on going about it. It's even better knowing changes to the framework could benefit all the similar applications I will be building (there could be a dozen).

Where do I start? Any best practices? Any pitfalls or areas to watch out for?

EDIT: I should clarify a bit, I'm not doing this for anyone but myself. I'm developing similar applications for different industries so the core is 90% the same, the differences are in the details.

Answer 1

Put all your code in a pile next to your desk. Start from a blank page, writing the code you wish you could write using your desired framework. Then use your existing code for components, and write the framework that would let you use your desired code.

Answer 2

I've done this before, while it was a fun and rewarding experiance I would have to say I would NOT do it again. Here are my reasons...

1) people expect you to support the framework. This was the number 1 challange I faced. Poeple came to me for enahancements, questions and bug fixes. Well this was really tough because I had moved on to other projects and did NOT have time to support them.

So make sure you have a clear support plan in place for the future. How will the application be maintained? Who will maintain it? How much time will be required to maintain it? And MOST important... who is going to pay for this maintenance?

2) developers hate to be told what to do or constrained. The framework I wrote simplified Data Access Layer for VB developers. It created a nice easy to use set of routines that would work no matter what MS did to change out the DB access routines. Well certain developers didn't like to be told to use a framework so they would bitch and moan. Frankly I got sick of hearing it and it caused some ill will towards co-workers.

So make sure you got some thick skin and can market WHY people should use your framework.

Answer 3

See Extracting Rails from Basecamp for David Heinemeier Hansson's take on how Rails came to be, from its origins as the framework used in Basecamp. Given the popularity of Rails, you may learn a thing or two from how DHH pulled this off. :-)

(Well, maybe not so much a how, as a why. But still entertaining. :-))

Answer 4

Here's what we at Starling Software have learned from doing this for QAM and QWeb.

Our approach is to treat this as a refactoring job spread across all the projects using the framework or proto-framework. We separate the framework code within each project into something that's built separately, so that, e.g., src/myapp contains the application-specific code, and src/qam contains the framework itself. Each project has its own copy of the application-specific code, and there's also a separate project that contains just the latest version of the framework itself. When we identify something within a specific project that wants to be in the framework, we:

1. refactor within the project to generalize the code (based on our understanding of both this application and all the others using the framework);
2. refactor within the project to move the genealized code from the application-specific part to the framework part;
3. apply this update to the project that contains the "master copy" of the framework;
4. merge the changes from that master copy into other projects that are also using the framework; and then
5. refactor within the other projects to use that part of the framework rather than their similar application-specific code.

This requires a fair amount of discipline to move the changes around quickly. If you've just finished developing a new feature and are immediately bringing it into other projects, the integration is easy. Projects that don't get updated for some time become much more difficult to update to use the latest version of the framework. If you don't bring changes into the master copy quickly, you may end up with different (and worse yet, conflicting) changes in the framework copies within two projects, and the merge can become quite a pain. You definitely want to update any particular project to the latest version of the framework before you start changing the framework.

Doing this needs a certain amount of practical support in terms of tools and so on. You'll want a way to move changes back and forth between the various copies of the framework easily and quickly. We have a custom tool to do this (qu), but I would imagine that one could also use a revision control system, particularly a distributed one that works by moving patches around, to help with this.

Having a comprehensive test suite for each application helps greatly; I'm not sure I'd want to attempt this without that.

For sanity, it's important to keep the changes small. Again, it's all a matter of how hard it is to merge and move updates. If it's always quick, and something you've been working on very recently, things stay easy. The large the changes are, and the longer it's been since you worked on that bit of the framework, the harder things become.