To re-write or not?

Question

This is a strategic question. I have a large piece of software, web based, that handles probably around 50 to 60 different tasks. The software uses about 100 MySQL tables, and can probably be considered as 3 big modules targeting 3 different sets of audiences. It runs very well.

The software was started 5 years ago, and has gone through incremental improvements. I pretty much designed and wrote the system single-handedly. There are definitely places where the design is bad; and after 5 years I think I can definitely do a better job if I re-write parts of it. But we never did rewrite it from scratch, so some of the bad design is left there. Code works but design is not all pretty at all places.

The system was built on a proprietary software platform; platform provide strong security (unparalleled in 99% of other web platforms, jailed silos, forced parameterized SQL, row based ACL on MySQL data) and performance (resources grow as number of applications, not number of concurrent hits). Turns out we don't need the performance benefits, but it was a nice thing to use.

The problem I am running into now is this: for the good of the company and myself, I'd like to transition the system to the point where I no longer have to maintain it, where I can get someone else to come in and improve and build on it. It's been shown that it'd be hard to hire talent to work on this existing platform. From this perspective, it'd be easier to re-write it (say it takes a year) using Java and then have the system be maintainable by others.

We are a small non-computer firm, and it'd be hard to attract top architects to do the re-design. I personally am too busy with other things to do the re-design. We can get outsourced contractors to do it, but I worry about quality of the system architecture. The system is also fairly complex, so it'd be hard to get all the requirements right (most requirements are in my head -- although you can make the case that the re-design and re-write would force us to do more documentation).

I am leaning toward re-writing the whole thing, but have serious reservations. Any thoughts?

Thanks.

Answer 1

You should never rewrite a system if you don't have to. It will take much longer than to try and refactor the existing system. There are numerous examples of software companies that try and do this and have failed. (See netscape).

Read Joel's post on this. It pretty much sums it up.

Answer 2

Do not rewrite it all at once. It is probably easiest to find a junior programmer with talent that you can bring in and invest in. If, however, the current technology is just something you can't continue using (that happens, especially with proprietary choices, but even in non-proprietary), then transition in small parts.

First, realize it doesn't matter what you write it in, if you don't have the in-house talent to maintain the thing from people who know the software, having someone who can maintain and grow the system will always be a problem.

So bring in that someone who will own and maintain it. If for some reason you can't get someone willing to work on the proprietary platform, get someone who's job it is to transition to the new platform (whatever you choose to use).

Then peel off parts from the old system and redevelop just that part - and this is the most important part - put that part in production before moving on the next part. This has two advantages. First, if needs change in a different part of the system, you can still maintain it without impacting the scope of your new project. Second is that all of the little details that have to be fixed come in manageable chunks instead of having a massive overbudget project that grinds all new development to a halt.

And a final piece of advice - if you decide to ignore all of the above and rewrite anyway - make sure that your rewrite adds no functionality over what the existing code does, it just repeats the same functionality as the current application. That way you won't get lost in all the new requirements everyone is trying to cram in.

One last thing - don't ever, ever outsource this kind of effort - you will regret it. Two things will happen. The first is that it will go massively over budget. Consulting firms in this case are basically guaranteed to underbid. The second is that at the end, you won't know the resulting code or architecture well, which will put you in exactly the same position you are in now.

Answer 3

See http://stackoverflow.com/questions/1064403/when-to-rewrite-a-code-base-from-scratch for additional discussion on a similar topic.

I think the ultimate question is - would it be more expensive to maintain the system, or have it re-written (and then maintaining that?). If the system is as proprietary and locked down as you describe, I imagine it could be quite costly to re-write.

Answer 4

So the purpose of the rewrite is organizational risk reduction? You want the system in a more maintainable state, making it easier for the organization to hire talent to make changes. I think that this is a valid reason for a rewrite, but you still have to factor in new risks and the short and long term costs.

Run some numbers on what these contractors would cost. Then double it as a high end estimate for when things go really bad. Then add in the cost of having the system fail unexpectedly after the conversion when bugs crop up.

Is the maintainability of the system worth that? Could you not instead freeze that code base and add functionality via some new but external application?

Don't replace one risk (the working but difficult to support application) with a larger risk (doubtful results of contract work). From what little information I have to go on, this sounds like a risky and expensive proposition for no great reward.

Answer 5

One strategy that I've used with fairly moderate success is the idea of incrementally rewriting software. Basically, we obey the following principles:

1. Quarantine off all the code you determine needs re-writing, and keep track of what is in there. All new code should be placed in new assemblies from this point onwards.
2. If a new feature is required on "quarantined" code, require that this particular section needs to be rewritten before the feature can proceed. (We initially were a lot tougher with this, and had a "no change" policy, but in the face of a major bug, it's not appropriate to rewrite and delay the fix / risk introducing more bugs)
3. Whenever transitioning old code to new code, get as close as humanly possible to 100% unit test coverage, and test the functionality of the new code against the old code.

We ran into some headaches surrounding building up new frameworks for the "new" code, and with getting stakeholders on board that a 2 week project could turn into a month because of rewrites. As well, no matter what you do, there will probably be some old unloved corners of your code base that are still in the "old" model, but we found this struck a good balance between keeping code up to date and not forcing a 18+ month rewrite project that would hold everything else up.