Implementing large system changes

Question

If you're familiar with the phrase "build one to throw away", well, we seem to have done that; we’re reaching the limits of version 1 of our online app. It's time to clean things up by:

• Re-organizing code and UI
• Unifying UI processes
• Adding more functionality
• Building for the future
• Modifying our database structure to handle all of the above

What's the best way to make this transition happen?

We want to avoid throwing all of our users over to a new system (once it's finished) ... they'd freak out and we couldn't handle the call load. Our users run the gamut, from technically proficient used-to-write-software types, to those that don't know what HTML is.

Should we start a new "installation" of our system and move users over to it gradually after we ensure this new design sufficiently solves enough of the problems with version 1?

Should we (somehow) change each module of our system incrementally, and phase? This may be difficult because the database layout will change, resulting in having to tweak the "core code" and the code for several surrounding modules.

Is it common to have a set of trusted, patient, "beta tester" clients using a cutting edge version of an app? (The goal here would be to get feedback and test for bugs on a new system)

Any other advice? First-hand experience?

Answer 1

We just finished plopping a brand new CRM system on our users, and let me tell you it was a TERRIBLE idea to do it that way: It was extremely painful for my team and for our customers.

I'd go through every possible mean to do gradual releases, even if it means doing more work. You'll be grateful because you won't have to go through heroic efforts to get everything moved, and your customers will appreciate the ability to get introduced to the product a bit a a time.

Hope that helps!

Answer 2

The answer, I'm afraid, is it depends. It depends on the kind of application and the kind of users you have. Without knowing what the system is and the scope of the changes in the version, it is difficult to offer an answer.

That said, there are some rules of thumb.

Firstly, avoid the big bang launch. Any launch of a system is going to have problems. The industry is littered with projects where people thought the bang-bang launch was a great idea, only for teething problems to bring the launch to its knees. Cuil was a recent high-profile causality of the big-bang launch.

In order to make the teething problems manageable, you need to work with small numbers of users initially, then slowly ratchet up the number of users.

Secondly, the thing that you must absolutely must positively do is put the user first. The user should have to do the least amount of work possible to use V2 of the system. The ideal amount of work would be zero.

This means that if you pick to slowly migrate users from one system to the other, you are responsible for making sure all their data and settings are migrated. For example, don't do anything stupid like telling the user they must use V1 for all records before 12/09/2008 and V2 for all records after.

The point of releasing V2 should be making the users' life easier, not making it needlessly more difficult.

Thirdly, have a beta program. This applies even for Intranet applications. Developing an application is much like Newton-Raphson's method for finding the root of a polynomial. You make a guess of what the user wants, you deliver it to the user, the user provides feedback and slowly but surely each iteration takes you closer to the solution to the problem.

A beta program will help you find the root much faster than just foisting new versions on to people without time for them to comment on the changes. Betas help get your users on-board earlier and make them feel included in the process; the importance of which I can not stress enough.

Answer 3

It sounds like incremental re-architecture should be your agile buzz-phrase of choice.

I've never done it on a web application, but I have been through some fairly radical client application changes that were done incrementally. If you invest a little bit of time up front to make sure that pieces of work are sequenced in a fairly sensible way it can work well. A small investment in good refactoring aids will be very helpful if you don't have them already. I can personally recommend jetBrains Resharper if you are using .NET, and if you are Java-based I believe IntelliJ IDEA includes similar functionality.

Answer 4

I agree with Esteban that gradual releases are best. It's like remodeling a house: getting it over with seems like a good idea initially. But means you have to plan everything upfront, hire a bunch of contractors and move out. Then something changes in the plan or a contractor disappears, and all that time you hoped to save is gone. Meanwhile, gradual change gives everyone a chance to stop and think between steps. Sometimes you can avoid later changes when earlier changes work out better than you planned.

I work on a system that had a huge scaling problem. We made a list of all the changes we thought we'd need and prioritized them by probable impact. Then we started making one change at a time. About half-way through the list, we found we'd solved the scaling problem. I still have the list, but I may never need to finish it. I'm free to add features and solve other problems.

Of course, there are times when it's best to bit the bullet and tear the whole thing down. But that's a lot less common than people tend to believe. And for critical operational systems, the "tear-down" decision can be fatal. Look at the big government projects that everyone agrees have to be brought to the modern computing era, but can't because some vital service will be lost. If the philosophy had been gradual change, maybe they would have been modernized one piece at a time.