Untangling a software mess, time for Big Rewrite?

Question

I recently started a job knowing that the company's practices needed help: no source control, no bug tracking, definitely no automated testing. They told me that their application code wasn’t good.

What an understatement.

Their code is an unholy mess of classic asp, non object oriented vb.net (hello recordsets!), embedded stylesheets, etc. The database is a nightmare: 500 odd tables, half of which appear to be old garbage, hundreds and hundreds of undocumented sprocs. The sprocs are full of business logic, ui logic (lots of PIVOT commands), etc.

There are four overlapping mission critical web applications that use this mess, and needless to say, most of the work was done by outside consultants who no longer return the company’s calls. (Who can blame them?)

Management is baffled at their web team’s inability to deliver on business requirements. I have had some success in explaining what went wrong: “imagine a ball of duct tape and wires seven stories high…”

Naturally, they want answers, not complaints. I considered a Big Rewrite, but that seems fraught with risk. I’m being encouraged to make a big win early, but I’m not sure where to start. I would appreciate any suggestions that y’all might be able to offer.

I’m thinking of approaching it like this:

• EDIT: Get TFS 2010
• Kill all "classic" asp—if it can’t be compiled, we can’t even be sure if we have all the references right.
• Find who uses which sprocs/tables via text search, delete unused stuff
• Rewrite functionality in pieces- e.g. replace the asp shopping cart with something OO.

Or alternately, we just keep slapping duct tape on the existing mess while building the One True System on the side.

I would appreciate any suggestions/war stories/offers of alternate employment, etc.

Answer 1

I don't think you can keep this mess and work with it. But I suggest to try to create Facade to this old garbage code until you rewrite it all. And then rewrite parts of it part by part.

Creating a Facade should not take much time (at least not much in comparing to Big Rewrite). And after that you will be able to immediately start improving software using clean new API. And when you will have free time - you will rewrite old garbage part of code using your way of coding.

Of course, it could be impossible to create a Facade for the type of code you have. Is it?

Answer 2

Duct tape works, and it's not easy to rebuild a 7 story building. IMO, I would keep slapping on the tape while planning a new system on the side if the boss is willing to pay you for the massive amount of hours it would require. Often I find management wanting to see 'progress' more than anything, in the sense that a shoddily coded but flashy new feature will glean many more props than a complete system rebuild that in the end appears the same. Boss probably doesn't care much about an OO cart vs. a non-OO cart. It's not fair, I know, and in an ideal workplace it wouldn't be the case. Just my 2 cents. Not difficult to set up a git or svn repo, though, and that is a step in the right direction. RT or Trac ticketing systems may be useful for bug tracking and feature requests.

Answer 3

brian - just keep slapping on the duct tape and scanning linked-in for new jobs :).

seriously tho', you need to actually present a report that details all associated risk and expense vs opportunity. in other words, you know how it should be done, so you'll have to prove that via a fairly comprehensive report detailing pretty much what you've said above. not a happy thought, but one that will actually be a product delivery as such, as the type of analysis that you're needing to do is a product that many companies would create a budget for.

hope this helps

jim

Answer 4

The best article I've seen on this topic comes from Joel, and it's here.

If you remember the netscape browser, this was one of the most popular browsers back in the late 1990's, but as Joel points out regarding netscape's downfall:

They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

The thing to remember is that no matter how bad that code may be, no matter how much of a trainwreck it is, there are parts of it that are probably very important to the application. If you scrap it and start from sratch, you risk losing critical functionality, and your application could be another netscape story.

It's never pleasant to go through and figure out what somebody's else's code is doing, but that is usually the safest and surest way to make sure you do a rewrite the right way.

So re-factor, improve, consolidate, upgrade to new technologies like ASP.NET, eliminate dead code, etc. but be sure to remember to learn from the lessons of the netscape developers, namely, don't throw away the existing code.

As someone once said, it's always less painful to learn from the mistakes of others than from your own mistakes.

Good luck!

Answer 5

I've found myself in similar situations but probably never anything that bad. My best advice is to do something that will be visible right away. Then take the time to needed to do some of the behind-the-scenes work. Also I suggest any page that you update with new code you also update with a new look - even if its only slight it will set the old and new apart. Hopefully this will get management on board with your plans to bring everything up to date. Good luck!

Answer 6

It's hard to know what to do in this kind of case, as neither keeping the old system nor rewriting the system is very appealing. One problem with rewriting is that there is likely a lot of knowledge embedded in the system- weird special cases, etc. And if the code is really terrible you are likely to not be able to easily extract that knowledge if you rewrite from the ground up.

One alternative to a ground up rewrite is to do what I call "George Washington's axe." It's like the old joke about the guy who has the axe George Washington used to chop down the cherry tree- of course he's had to replace the handle three times, and the head five times.

When you need to add features or make changes to the system go in and first add tests that cover as much of the functionality of the part you need to modify as you can manage. Then refactor the parts of the code that you need to be able to understand in order to add the feature or make the change, running the tests at each change. It's also not a bad idea to run any functional tests you have as often as is practical. Eventually you will have rewritten the entire system, or at least have walled all the ugliest stuff away, but you won't have had to stop all development for an indefinite period of time to rewrite from scratch.

Of course there are some downsides to this approach- features and changes will be slower than they should be for quite a while, until you get things under control. In the end you have to look at all the factors, weigh them and make a decision- it might be that rewriting from scratch is the best option.

Answer 7

I highly recommend you read "Working effectively with legacy code"

Incremental rewrites and step-by-step de-crufting is the best solution if possible. Much as we (as techies) would love to down tools and fix the ball-o-mud, most of the time you can't stop delivering business functionality for months on end - if you do you're going to lose the goodwill of the end users for the new system before it's even completed.

Answer 8

Brian - my sympathies, it sounds like you've taken over one of my old clients. Sorry for the mess I left you, but I did try to get things cleaned up before the client ran out of money.

Seriously, I had a client with 25+ apps in a similar mess - after two years, about 1/2 of them have been reviewed, and the top 5 cleaned up a bit.

You don't mention the size of the team, but overcoming reluctance to change things may make it impossible to do anything. If that's the case, run away - there's nothing more frustrating that to re-write something just to find that your other developers are sabotaging your efforts by not being onboard.

Assuming you have support, you can look at solving this from a top-down or a bottom-up perspective.

The first thing to do is put some structure in place: get the whole mess into a version system, so at least you can track what you're doing, and roll back if needed. Set up a separate testing environment so you can try things without affecting the on-going development and ability to support the system.

If you're approaching this from a top-down perspective, you'll want to start by cleaning up the UI elements: strip out all the embedded CSS, clean up & rationalize it. Inventory the pages, dependencies, etc (this alone may take some time, especially with classic ASP). Identify repeated code blocks, and consider moving them into shared user controls. See if you can map the UI components into database elements.

Analyze access logs to find what's being used, and hopefully identify code that's no longer needed. You'll probably find a number of errors, too - these can actually be quite helpful in understanding the system. You'll probably find that a small subset of the application gets used more often - that's good place to look for an easier, early win.

Create a set of small, clean CSS files, and create some master pages you can use to standardize the look & feel. You'll have to re-write the asp pages, but you may be able to update webforms pages to use the new master pages more easily.

The goal of this approach is not to fix much, but to understand the structure and impose some organization on it. Low-tech can help: print out every page on the most common applications, and tape them up on a wall. Seeing everything laid out like that can be a big help.

The bottom-up approach starts at the db: inventory all your tables & stored procs. See if you can figure out where & how each is used. You may think something isn't used, but really the only way to know for sure is to remove the object from the database (or rename it) and do testing. (Yes, I've seen code that dynamically builds table names based on user input - no analysis tool is going to help with that.)

Once you get a better idea of the dependencies between the different parts, you can start planning how to gradually do the fixes. If you can isolate different subsystems, consider doing a migration to a new schema/set of tables, and build a proper data access layer. You can consider incorporating unit testing at this point, but honestly you're going to have enough problems on your hands, and this might turn out to be a deal-breaker for people unfamiliar with it - learn to walk before trying to fly.

If you can't completely separate parts, consider still doing the migration, but see if you can put into place views or new sprocs as your facade layer until that code can be re-written.

There are a number of books dealing with re-engineering legacy systems, I'm not sure that any one of them is going to deal with this situation, but it couldn't hurt to review the literature for additional tips.

Good luck, and don't forget to keep the resume updated!

Answer 9

Tackling just the database part of this, here is a book that wil lhelp you out: http://www.amazon.com/Refactoring-Databases-Evolutionary-Database-Design/dp/0321293533/ref=sr_1_1?ie=UTF8&s=books&qid=1268158669&sr=8-1

You have to set thing up properly to refactor a database and this book is invaluable.

There are also tools that can help you find the most poorly performing queries, start with those. If you can get a big win in performance on something that is annoying everyone, then it will give you more leverage to keep on fixing the other problems.

You might also look at a book on performance tuning specific to the datbase backend you have. There are a lot of know perfornace issues that relate to both database design and query design, knowing how to fix them can help you out immersurably as you refactor this mess.

I know it's tempting to just throw it out and start over new, but you will be introducing major new bugs that way, taking a huge amount of time while the users see no improvement and possibly missing some very important business rules that must be enforced. While incremetally changing and refactoring appears to be harder in a case this bad, it really is the better choice.

You might talk to the users and find out what they perceive as the worst problems the system has, that's one place to start after all making the users happier is part of what you are there for.

Make sure that you document the performance improvements and other changes for your resume. Think how much better it will look to your next potential employer when you can give actual performance improvement figures. Actaull accpmplichments attached to figures that show how much you accomplish are rare on resumes, this job can really make you stand out in the future as someone who gets things done.