views:

2110

answers:

17

You've worked somewhere for 3 years. At the beginning of the second year you were involved in what you thought would be a small project. You foolishly decided to take some risks with the choice of tools, you got slightly out of your depth, you ended up writing some nasty hacky code in the end as delays kicked in.

The client loved it, your boss loved it so much he started white-labeling it, and selling it on to other clients.

You've been the lead developer on this code that you can't bear looking at for over a year. It goes against all your programming morals. You are scared of refactoring it because it's been live for ages, everybody is happy with it except you. You're not quite prepared to squeeze in the refactoring in work time, because there is always more important things to be done.

Do you take it upon yourself and decide that it must be refactored, let it eat into your spare time because you deserve it?!

Do you carry on and hope nobody else notices what an awful mess the project is?

Do you leave your job and hope your work friends don't phone you, curse you forever more?

+25  A: 

"You are scared of refactoring". That's the key here - don't be. No one noticed yet, it is all working, take the time (your time) to get it where you think it should be. Start carefully with source control as safety net and start digging out of the hole.

In my mind there is no better way of becoming a real architect than that. Good luck.

Otávio Décio
Regression testing is cheap insurance. If you don't have an automated test suite, do that first. Then there's no reason for fear of refactoring.
S.Lott
Some code is more suitable for [unit-]testing than other code. If it is a distributed system, or a system, the state of which is ever-evolving, or just a weird bunch of stuff glued together + non-automatable GUI, and you are looking at something that is easier to test manually :(
Hamish Grubijan
@Hamish that is not really true. Just because it can be hard to automate doesn't mean that the manual solution is easier - what is your definition of easier? Less costly, quicker, less steps? SQLite is an application with no GUI, networked, embedded in dozens of different environments - and it is **heavily** tested to death http://www.sqlite.org/testing.html
matt b
@matt b, SQLLite is a product that can be used by millions, potentially. Having no GUI helps. Also, while its state can evolve, it is easy enough to isolate use cases and start from empty db. It is also easy enough to combine two simple use cases into a more involved one. You are right though - what I said is not always true. I should have qualified it with "sometimes".
Hamish Grubijan
I think it should serve as more of an inspiration for what is possible as far as automated testing and having high standards rather than an edge case. When faced with setting up test harnesses/etc for an app that is not currently easily testable it might seem easier to just test it manually, but that path can lead to costs that add up over time; automated testing might involve a larger initial time investment but likely would lead to a huge return on investment.
matt b
+3  A: 

You need to be able to refactor any code you touch. It should not take a lot of extra time to do so, If you do not have time at the moment add some comments on what you would like to do within the code.

On another note: What is your career goal ? Do maintenance work on this until it is useless ? I would tell my boss that you want to move on to other projects. If not you will end up being the maintainer of this one project until it is no longer useful. At that point you will be so antiquated that you will become useless to the company you work for.

Another option - move to management :)

Romain Hippeau
An average software project is useful for about as long as an average hard-drive. However, some companies which grew too aggressively, can end up maintaining 10+ years old projects. How soon something will become obsolete depends on the company, the tools they chose, and the business they are in.
Hamish Grubijan
+20  A: 

You are scared of refactoring it because it's been live for ages

So what you're saying is that there aren't any unit tests.

If you have sufficient coverage and high enough quality tests you could refactor with some confidence. As long as you moved slowly and didn't break any tests you'd be fine.

Start by generating unit tests. Create that safety net before you start.

duffymo
I don't down-vote often, but sheesh. Your advice here is like telling a skin cancer victim to wear a hat.
MusiGenesis
So where's your great advice? I don't see anything from you. I don't think it's worse advice than "go for it". At least it's constructive. Besides, writing those tests might suggest a better design, because hard to test is usually a good sign that refactoring is needed.
duffymo
@MusiGenesis - the point isn't unit tests for the sake of unit tests, but rather unit tests to make him more comfortable editing a live product.
Matchu
I've added my "great advice" below, although it has absolutely nothing to do with programming and everything to do with intra-office politics. Regarding unit tests, I assume you guys are advocating TDD without going to the extreme of asserting that it is not possible to write good code *without* TDD, in which case I would say that your championing of TDD is irrelevant to the asker's question.
MusiGenesis
Nope, not saying that it's impossible to write good code without TDD. I'm no zealot. But I'd say they help. What's the alternative - writing without testing? Leaving it to QA?
duffymo
I've got thow in my two pennies and side on the unit testing here. It can help to not only to give you confidence that your changes aren't going to break anything, but also clear up what exactly is wrong with the current version and where the fixes are needed most. Starting from scratch can open a huge can of worms as you become to focused on making "better" code and end up building a rocket ship to get you to the shops. Convincing the bosses can be tricky but you can always ask for time to verify that the code is working as expected, from there it's a short hop to say "this needs refactoring
Dylan
I'm surprised no one has mentioned the book **Working Effectively With Legacy Code**. It calmly addresses the fears / objections that come out when automated testing is suggested on this type of project, and guides you through slowly introducing unit tests and improving the code at the same time.
Patrick McElhaney
I've down voted you for one simple reason: **unit** testing is testing units, not the architecture, which is the primary target of refactoring. Refactoring almost certainly will break unit tests, as most of them do rely on implementation in the real world. What he needs are automated tests, not unit tests.
HeavyWave
You have to start at the lowest level, but you need not stop there. Integration tests, system level tests - of course they're all necessary. And of course they should be automated. This could mean an Ant build.xml that's kicked off manually or a Cruise Control driven automated build. Only downvotes from you, HeavyWave, and no other entry. Perhaps your sage advice would have been more accessible if you'd put yourself out there and offered it as an answer.
duffymo
@HeavyWave, that argument doesn't make much sense. Unit tests guarantee that all the individual unit tests are working as they should be. They allow you to have confidence to proceed to testing higher levels (automated integration tests, etc). There is little value in testing the higher levels if your low-level units do not work. Refactoring might break unit tests - of the unit you are refactoring! If it breaks other unit tests then you have design issues. And as duffymo says what is the alternative - no unit tests at all?
matt b
@matt b, "There is little value in testing the higher levels if your low-level units do not work" same applies to unit tests. "If it breaks other unit tests then you have design issues." If there were no design issues refactoring wouldn't be necessary in the first place, would it? The alternative is to test what actually matters, testing the smallest units very often does not provide any assurance that the whole system is going to work, and even regardless of that, you will have to spend time writing unit tests **and** refactoring them as you refactor your code.
HeavyWave
+4  A: 

I'm an idealist - I'll always side with re factoring. As long as you are careful and have good testing mechanisms in place you should be able to very gradual refcator the code fairly safely. The key is to make gradual changes rather than sweeping changes.

If you don't have reliable unit testing then work on that first - you should be able to make changes to your code and still sleep comfortably at night!

You should also consider that if everyone else (including clients) think its so awesome, perhaps its not as bad as you think it is. I've found that we programmers tend to be very idealistic about clean code and architecture - its easy to lose sight of the fact that what really matters is the compiled software.

Of course this is all only relevant if this is what you want to be doing - don't feel tied into maintaining a piece of software you wrote out of guilt.

Kragen
To the last part, usability and features does not mean the code is good.
Razor Storm
Yes, but awesome code that never sells is like a hot woman that never gets pregnant.
Hamish Grubijan
@Razor - Good code makes software easier to maintain, but in itself it doesn't make that software better.
Kragen
+4  A: 

I think the question becomes: is it a maintenance nightmare?

If it works, the client is happy and it's not hard to maintain, has a crime been committed? Yes we like to structure it properly, but we need to make sure that's to deliver better ROI to the business and not to satisfy our own OCD.

Definately do NOT refactor it in your own time. If you've done the best you can with the constraints imposed on you by the business, how can it be your "fault"?

SteveCav
+6  A: 

I have just 50% re-factored a monster child of a very brilliant programmer (he moved on to greener pastures); his problem is that he was usually stuck in the first gear - making progress very fast, being able to keep a lot of stuff in his head at once, but not paying much attention to details. If he was a soccer player, he would be the guy who scores the most. I would be a defender I guess.

Not trying to sound cocky, but I think I was the perfect complement - someone who prefers to work in second gear, loves small but sure changes, loves unit-testing, etc. If only I had more time, I would polish this bugger until it shines.

I guess it might take a programmer with certain personality - loves to simplify, like to move cautiously, someone who over-asserts and over-logs (within a reason).

The thing is that before you build something, you do not know what it should look like exactly. A working prototype is better than any spec. You might want to have someone else polish it. Their job is easier - they know what it should do, they just need to remove rough edges. Plus, a fresh pair of eyes is always good. As long as that someone is as good coder as you are - you should be ok.

I usually try to hold myself to high coding standards, but some of my colleagues are more relaxed than that, so I have less of perfection paralysis now as well.

But then again, it could be you who re-factors it. Your problem may be that you know the design all too well.

Hamish Grubijan
Carmac (the guy who wrote the Doom 3D engine) had an older, more experienced programmer who cleaned up and optimized his code for him. A good company (I haven't worked for one yet, apparently) recognizes that different people have different strengths and weaknesses, and puts people in roles where they can succeed.
MusiGenesis
Sounds like a winning combo.
SteveCav
@SteveCav: who, me and Hamish? He talks too much. :)
MusiGenesis
Hamish Grubijan
@MusiGenesis, Trader Joes had great wine on sale tonight, and I did not have anything less than $20 on me.
Hamish Grubijan
@Hamish: I think you can get 8 bottles of Trader Joe wine for $20. You're gonna have a helluva hangover today.
MusiGenesis
@Hamish: Thanks for this answer, how do you feel about him leaving you this monster?
Dr. Frankenstein
I liked it. It was a 30-project solution in C#, which was fun=quick to work with. Surely beats fixing bugs in MFC-based C++ GUI. I was not warned of all design pitfalls, but I had fun discovering them. For someone who is not quite a senior dev. yet, it was a good project to establish confidence. It is an existing product that is important to the business. All I have to do is make it better and not f. it up. In fact, I did not have to re-factor it; I could have just added features. But, I feel in love with this duckling, so padded my estimates a bit. The orig auth wrote lots of good stuff too.
Hamish Grubijan
+3  A: 

If you're the lead on the project and it's your only responsibility you should appropriately schedule time to re-factor parts of the project (it doesn't have to be done all at once does it?..). If you have more responsibilities that you are not the lead for you should explain to your CTO why the re-factor is required and again schedule the time. CTOs/CEOs understand technical debt.

You should also start by just doing unit tests (I assume the quick hacky codebase didnt include full unit tests :P). Make sure that your re-factor doesn't change behavior.

Ken Struys
+1 good points here, not heard of technical debt before
Dr. Frankenstein
it's a good way to justify testing to business people
Ken Struys
+1  A: 

I've been in this situation many times, both because of crap I wrote myself and because of crap that other people wrote and then dropped in my lap. My advice years ago would have been to use your powers of communication and persuasion to get your bosses to understand that it would be cheaper in the long run to toss your duct-tape-and-chewing-gum-special in the trash and start over.

The reality is, unfortunately, that your position is hopeless. If your bosses had any technical capacity whatsoever, they would already know what a piece of junk it is. Since they don't know that, you instead become a person who is insisting that this valuable program (which they've already made money off of and which has already substantially enhanced your boss' standing in the company) is in fact worthless.

If you stick to your guns and insist that the program should be rewritten from scratch, you create a situation where the company has two options: 1) take the hit of the rewrite; or 2) allow you to resign so you can "spend more time with your family".

I have to vote for your second choice (carry on and hope nobody notices), because it's in everybody's best interests (except your clients, of course) to not notice.

MusiGenesis
+1 Joel Spolsky, for instance, is very much against rewrites as well. There are different IT managers - some of them still write code 30 hrs per week. On occasion you will have a manager who is more anal about code quality than you are. But, most commonly, there will be a mix of someone with bachelors in business + MBA and someone who is somewhat technical, but concentrates on the business sides of things. Either way, there are 2 challenges: A) Find time for improvements without anyone noticing :) B) Figure out what needs to be done first, second, third, and how. I think that A) is hardest pt.
Hamish Grubijan
I laugh every time I read that Joel Spolsky column. He has definitely not seen the things I've seen out there in the wild.
MusiGenesis
Yes, I bet he never worked at places which score less than 9 out of 12 on Joel's test :) - that makes him an elitist.
Hamish Grubijan
"this valuable program is in fact worthless" It's clearly not worthless if people buy it. It's just has a big amount of technical debt. As with every debt issue, you have to start finding where to save to pay up, the bosses should be supportive of this issue if they could understand it. If they can't, then you have to slow new features down, or increase the amount regular debugging would take to start paying up. Your rewrite solution is like moving out of the country due to your inability to pay the debt.
Vinko Vrsalovic
@Vinko: if people buy your software but you then have to spend more than what they paid to maintain it, then the program is even worse than worthless. My rewrite solution is more like declaring bankruptcy than moving out of the country.
MusiGenesis
Your assumption that you spend more maintaining it than what was paid for is not warranted in the question, the question states that he can move on but would strongly prefer better code. My impression from the question is that he is just indebted, not bankrupt.
Vinko Vrsalovic
The asker describes his code like this: "awful mess", "nasty hackish code", "code that you can't bear looking at", and "goes against all your programming morals". You're trying to summarize this as "he can move on but would strongly prefer better code"? I'm sticking with my bankruptcy analogy.
MusiGenesis
Not a single phrase among them mean that it costs more to maintain the mess than what is received for it. So I'm sticking with the debt and that the program has a positive value. Also, we programmers usually tend to take a harsher view on our code than what it really deserves, so I usually take that dramatic phrases with a pinch of salt. That is not to say that there surely are some really dramatic cases where bankruptcy is required, but I'd argue it's not the norm and that you should try to fight it first.
Vinko Vrsalovic
+1  A: 

You should learn to master the skill to sell small refactoring improvement in the feature package. I'm not saying to lie to your bosses or trick them, don't. Just tell them that the feature "A" takes At + Rt time where Rt is the refactoring time (a small percent added to the feature "A" time. I second every good advise above about having either automated unit tests or regression ones. I duly note that writing unittests after is quite boring (even more if the code is actually yours). I'd start with solid functional/regression tests for the overall functionalities and picking up the single unit of codes with not-so-deep unit tests. What I mean is: do not spend the whole lot of your time writing tests for each and every case during the refactoring session.

Keep in mind that also the tests needs to be refactored.

Lawrence Oluyede
By the way, in the world of C# there is Re-sharper, StyleCop, FxCop. Having these stable tools help you fix-up your system is a good way to start - it can get you 20% toward the goal with relatively little effort. It can cost some $ though, and some languages/platform have better tools available than others.
Hamish Grubijan
Assuming you know the value of `At` :) In my experience, it's 3 times bigger than you expect, even when you take this rule into consideration.
Mark
@Hamish: Even with Re-sharper the refactoring is not magical, pushing the "extract method" button does not move the finish line anywhere nearer without a clear refactoring path. That takes time and time reflects on $. What I've learn in these three years working on a gigantic codebase in my company is that most of the time small refactoring leads to big (not anticipated) changes so it should be done carefully. On the other side that kind of problems arise when many people are the owners of the source code and we already know that not everybody adheres to the same standard or quality of code.
Lawrence Oluyede
@Mark: eheh you got me. I learned one rule from one of my senior colleagues: to overestimate, at least a bit. With the time that "bit" is more and more precise tending towards zero but if someone is already planning for refactoring is better to overestimate a bigger bit.
Lawrence Oluyede
@Lawrence - there are many "truths" - it all depends on the exact situation, as well as personality, I must add. I have seen people approach a task very differently. I personally get distracted by little things easily, so it helps me to remove all traces of Hungarian notation and other obvious crap before I can start thinking about the big picture. Also, when I start reading a new book, I do not skip the acknowledgement pages and other mostly useless stuff just in case I miss something important. I know other people who read very diff-ly. Yes, refactoring is an art, but partly is mundane work
Hamish Grubijan
+3  A: 

As you go through, refactor what you're working on anyways, and realize that:

  • It's getting better everyday; your drift is positive.
  • Most professional code is a hack at some level.
  • You've learned some damn good lessons along the way.

If you can't hack it, find a new job; three years is a pretty long stint in technology. You'll take lessons with you, you'll get a raise for doing it, and you'll get to work on a brand-new project.

As a strong suggestion, spend at least a month documenting every last hack you can remember, so that you're not leaving the next guy a terrible mess; do not burn your bridges with your replacement.

I'd mix the strategies; refactor what you touch to be right. Do not spend overtime on it, but slow your estimates down to allow yourself time to do things in a sustainable way. While you refactor, if something can't be refactored right then - or at all - document it heavily. In two or three months, start looking for new work, and take at least a month to transition out of the role when you do find new work.

Dean J
This is a good advice, assuming that there are good objective reasons to move on. It is also possible that the asker is mildly dissatisfied with his code, and has had only 6 hours of sleep due to an asshole neighbor who has a car with a sensitive alarm, and his air-conditioner broke, and he is a little grumpy at the moment. Of course, nobody would know better than him, but in my experience, there will also be something you do not like, a co-worker that you did not click with, a project that was a bit too stressful. Deciding to leave should be a well-calculated move. Weekend is a better time to
Hamish Grubijan
Bah, by "will also be" I meant "will always be" ...
Hamish Grubijan
@Hamish Grubijan: deciding to switch positions should be a several week - or longer - decision; never, ever make that one in a hurry. Absolutely agreed!
Dean J
+11  A: 

Do you take it upon yourself and decide that it must be refactored...

Yes!

... let it eat into your spare time because you deserve it?!

No! You don't deserve it. You've done a great job. Now, continue doing a great job by explaining how design is a learning process, you've made some mistakes, and you need to go back and do some housekeeping so that the project can flourish.

Trying to be a hero and silently burning yourself out is not good for your company and not good for your career.

Patrick McElhaney
"Trying to be a hero and silently burning yourself out is not good for your company and not good for your career." This bears repeating. Ultimately it's up to your company to decide if there is value in refactoring. Most programmers get better over time and cringe when they look at their earlier works. Unless you are the type that never gets any better, everyone is eventually confronted by the desire to refactor.
Bill
+1  A: 

From the description of the problem it sounds to me not that the author is scared to refactor but that he knows intuitively that refactoring while continuing to add features might be biting off more than he can chew.

Refactoring a large project like that will take some time and thought and a lot of focus which isn't something that can be easily done while continuing to fix bugs and add new features.

I'd suggest talking to your boss and convincing him that a refactor is needed and you will need 3-6 months to do it. You could also suggest getting someone else on your team and split the code into two branches, you can work on refactoring while your new team mate(s) adds additional functionality.

Basically like any difficult problem its best to divide and conquer.

Remember this is a successful product for your company so I wouldn't think of it as a failure in anyway. This happens all the time with small projects. Anything that grows seriously beyond its original scope is going to need some deep refactoring.

Jai
+1  A: 

You are scared of refactoring it because it's been live for ages, everybody is happy with it except you. You're not quite prepared to squeeze in the refactoring in work time, because there is always more important things to be done.

If everybody is happy about it, then don't touch it, unless you're getting paid for refactoring.

Do you take it upon yourself and decide that it must be refactored, let it eat into your spare time because you deserve it?!

No. You know what should be top priority for a good programmer? family/relatives/wife/girlfriend. If you hate the project, you definitely shouldn't do it in your spare time. Your spare time should be more valuable than the project.

Do you carry on and hope nobody else notices what an awful mess the project is?

I would wait till they notice and ask me to refactor it. Once I'm done, I would ask for raise.

Do you leave your job and hope your work friends don't phone you, curse you forever more?

Now that sounds plain paranoid. I'd recommend to take a vacation. Friends are called "friends" because they're supposed to understand that you hate this thing with passion.

Leaving your job because of "monster you created" makes sense only if you really hate that project, and there is no other way to deal with the problem. Few things you should think about:

  1. "Don't fix it if it ain't broken" if code works and everybody is happy about it (even if it is a mess), do not refactor it, unless asked. Once upon a time I worked on a project ported from a very old code. The code was horrible, so I boldly decided to fix it (project had 2.5 megabytes of c++ code total). Right now I think it was a really stupid idea. Modification of code introduced several new bugs, prolonged development time, and hadn't improved a thing. Code looked better, but that is it. I.e. it makes sense to avoid refactoring unless it is required for fixing current problem. Because (according to my experience), a programmer can waste insane amount of time "refactoring" - i.e. modifying code that it looks better, without improving program functionality.
  2. If you think that code should be refactored, it doesn't mean that you should do it. Try to politely tell your boss that code needs general cleanup, but you're too busy fixing bugs for the project, so it would be really nice if some other guy (or several guys) started cleaning up the code according to your guidelines. You hate the project, the "other guy" may be indifferent about it, so he'll have more enthusiasm than you. Just make a git repository, create two branches (one for stable release with bugfixes and other for refactoring) and start working.
  3. TO me it looks like you're thinking backwards. You created valuable asset for the company, which is liked by people, and can be sold. And instead of being proud about it you waste your time worrying about "bad" code quality. End user won't see the code, he/she needs working product. User doesn't care what product's source code looks like, as long as the product works without bugs. Useful software isn't guaranteed to have a nice source.
  4. It is possible that you could get out of the project without quitting the job.
SigTerm
A: 

Answer for me is always: rewrite. Rewrites are fun because they reflect on the knowledge you've gained since you last started working on the project, and they also give you an opportunity to make the code as clean and efficient as possible.

Zanneth
+2  A: 

I would highly recommend Michael Feathers' book, Working Effectively with Legacy Code. It addresses exactly this issue - how to take a large, unwieldly, badly-written codebase, and clean it up.

The point that Michael makes over and over is that you have to get your code under test, wherever it's reasonable to do so. He gives refactorings that you can use to help introduce unit tests to clunky modules, and break dependencies. You can start to build unit test coverage for small, safe pieces of your code, as you go through. Create unit tests covering a section of code as much as is feasible before you change it, so you can determine whether your changes have unintended consequences.

RMorrisey
+2  A: 

There is a good book about this, Working Effectively With Legacy Code. Where legacy code is defined as code without tests.

The gist is what the scouts say when camping, "leave the site cleaner than your found it." You don't have to boil the whole ocean at once, which would probably break things anyway. If you see a quick fix make it. As you add or modify put a test in. As the tests build up and you start to pull out duplication,the changes can snowball and it will become less intimidating and of higher quality.

At the highest level it's working, don't mess with it unless you're sure you can make it better or you need to dig in for some other reason.

Paul Rubel
+1  A: 

Do you take it upon yourself and decide that it must be refactored, let it eat into your spare time because you deserve it?!

Spare time? What's this?

I've inherited several monsters over the years. I sometimes describe them as houses of cards made of tissue. Every now and then, I buck up, look at one, and think "If X did this, how would he/she do it?" with X being the appropriate role model for the job (coworker, Dijkstra, etc).

A lot of the refactoring can be done by encapsulating current functions into separate units. Sometimes, much clarity can be gained by simply restructuring what seems to have grown haphazardly (new namespaces in C# come to mind).

When I feel guilty over time potentially lost refactoring, I tell my self: "No, at least I made it simpler."

MPelletier