What are your best-practices for bugs that are hard to solve?

Question

I sometimes find myself trying to fix a stubborn bug, only after a while figuring out that something quite obvious was wrong.

This happens to me when I think the problem is related to something and I kind of get 'blind' for other things that can be wrong.

Most of the times it helps to have a night of sleep and the next morning you see the problem immediately.

Things that happened to me in the past:

1. Editing a copy of the real source file that did not have any effect.
2. Not focussing on the real problem, but trying to figure something out while the real problem was already solved.
3. Not doing a compile/build as you were working with interpreted languages before.

So what are your 'blinding' experiences during bug tracking?

Answer 1

One thing that helps a lot is to find a good balance between focussing and not obsessing.

• Make sure you can work on the bug without being disturbed
• But also know when to take a break if you cannot concentrate any longer

And yes, there is a point where a good night of sleep is the best you can do.

Finally, discuss the problem with a colleague, often a second pair of eyes and point of view work wonders

Answer 2

Walk through the code with a debugger, line by line. At the very least, this should pinpoint the line where something you don't understand is happening. If your initial guess at what the bug is fails, stop guessing and use a debugger.

Answer 3

Use an editor that lets you set breakpoints

With Breakpoints, you don't have to start at the beginning of the code over and over as you debug, which makes it easier to step through the code

Answer 4

Like you said, the best thing is often to step back from the problem.. maybe work on something else for a while, or just take a walk.

Answer 5

Many times I have gotten caught up trying to fix a bug that really did not have to be fixes. In many cases, a user simply wants a particular behavior, regardless of whether that behavior is warranted or not. I have had situations like these where I will spends lots of time trying to "fix" the bug, only to come to the realization that the bug really isn't a bug, or is not practical to fix.

Answer 6

Using your version control system, go back in time until just before the bug appeared and diff that code against the first revision where the bug is introduced.

Use a binary search tree approach to make the process faster.

I'm told there's a git command or tool that makes this easy.

Answer 7

In 3D (games, for example) rendering stuff, debugging is always a nightmare. It sometimes goes down to pixel debuggers, which displays when and during what draw call that pixel was modified, from what to what color.

Answer 8

If your .Net project fails on Mono, debugging is a nightmare. Most of the time it's about Console.WriteLine.

There're a few options. You may:

• Debug on a linux machine. There's a command-line debugger for mono.
• Run the code in a linux machine, and remote-debug from visual studio using some remote debugging plugins (I didn't succeed at it yet)
• On windows, run MS .Net runtime but link it with mono class libraries. You can do it except for mscorelib, though it involves some messing around with the .net framework settings like disabling strong key checking for .net assemblies.

Answer 9

This question covers much of the same ground: http://stackoverflow.com/questions/366453/best-practices-for-debugging

Answer 10

debugging!

Answer 11

I agree with sleske but one thing I'd add is to break the unit of work into smaller units. So if it's bug in a method...make smaller methods. It will clear the fog of war.

Answer 12

Print outputs everywhere if you don't want to set breakpoints. Helps me big time when debugging. And then, of course when you see THAT variable output equals something totally different to what you thought it should, its like light breaking through.

Answer 13

Leaving it alone and calling your colleague to have a fresh look at the problem. Time and again I find this approach solves the problem in 5 minutes, while you could lose the whole day.

Answer 14

• Use the debugger, make sure you know what the 'expected' behaviour is, find the point where 'expected' behaviour is no longer happening
• A source control helps, in case when you can roll back to the working version of the code and start adding your changes back little by little, until you find that bit that breaks the functionality.
• Use the event log. I had a bug recently which logged a DllNotFound exception in the Application event log. That made no sence since the file was there all right. It took me some time to peek into the System log, and there was another error there about a dependency missing. So use all available logs. Write critical information to the event log yourself and make sure it appear as expected.
• Use google or stackoverflow to search. Most problems have been experienced by other people before, you might be surprised by how many people had the same problem.
• START writing a question to stackoverflow. For every question I asked there were at least one or maybe two that I did not, because just formulating the problem properly in writing led me to the solution
• If possible, create a small 'throwaway' empty project and try to replicate the problem there. That might help to find out if the problem is caused by your code or by other parts of the project
• Use the help of a colleague
• Walk away for a while if you can, or at least try to do something different. Your mind will keep working on the problem subconsciously, and may deliver a solution soon.
• If nothing helps, read the documentation and manuals. ( :-) )

An example where nothing helped: Once I was going crasy because my DataGridView would not show checkboxes in a checkbox column. There was absolutely no way it could be possible. Took me long time to find out that, for some reason, the checkboxes would not show if a column was less then certain width. They were still visible in the designer. (I think the reason was a certain UI toolkit we were using). I wouldn't find out, but just by accidental mouse move I resized a column at run time and voila - there they were, my checkboxes, all this time, sitting there invisible.

Answer 15

All good answers.

For really tough bugs, such as numerical routines that run for a long time and get slightly different answers from what they used to get, there is a laborious method that usually works.

Get one foot on each side of the problem and work toward the center.

Like get an older version of the program that got the right answer, and the newer one that doesn't. Incrementally make changes to either one to bring them closer together, still keeping one on one side of the problem, and the other on the other. Eventually, you find the one change that makes the difference.

Like run an older good version of the program under a debugger, and the newer bad version also under a debugger, at the same time. Single step them side-by-side until they do something different. Then repeat, homing in on the step at which they start to differ.

I've found this working-inward-from-both-sides method is applicable to all kinds of difficult problems, not just software.

Answer 16

I've found when staring at something that should be working but isn't, I'm often trying too hard. Grabbing a friend and explaining to them how it should work, line by line, bit by bit, usually causes the problem to jump out.

Answer 17

I have to be good at finding and fixing bugs when I program like I do. ;)

First of all, I have to admit that the bug is mine. When working with a group it is easy to blame someone else for a problem. When I get that inclination, I start by collecting enough evidence to prove that the bug is theirs. About half the time, I collect enough evidence to determine that it is mine. By that time I'm well on my way to a solution. Otherwise I approach the other developer explaining what my code does, what the response is, and what surprises me about that response. That makes it easier for them to either point out where my expectations were wrong or where their contributions may not be working correctly.

When I do determine that the problem is mine, I approach it with "the relentless application of logic." I collect all of the information I can about the misbehavior. Then I examine it and the code side by side, trying to figure out what could cause unexpected behavior. Usually this points to some part of the code that becomes suspect. At that point I try to instrument the code further, saving copies of variables or logging critical information. Based on the further information collected, I evaluate and refine my original hypothesis and go on. Several such iterations usually nail down the bug or point to a new area for investigation. I guess that boils down to analyze -> instrument -> test -> repeat. When I find the problem, I always go through at least one more instrument -> test iteration to make sure the bug really is fixed.

As others have mentioned, ask coworkers for help. That helps in two ways. First, you will understand your code better when explaining it to them. Even if your coworkers are not at your level, this can help. Second, they may see things you did not notice or ask questions that did not occur.

Answer 18

Something I've found helpful is to bear in mind that, when dealing with the really interesting/nasty bugs, you don't have to hit the right solution all at once. Anything that tells you something you didn't already know represents a step in the right direction, and thus gets you closer to the answer. Even figuring out what the problem isn't helps, because you know to look someplace else.

Suspect that maybe the bug may be caused by condition X? Then try dialing X up to 11. If the bug gets worse (or shows up more frequently), X may indeed be your culprit. If it has no effect, look elsewhere.

Is the bug intermittent? Then temporarily forget about trying to figure out why it happens and focus on when. If you can come up with a test case that reliably reproduces the bug, you're well on your way to figuring it out.

Think one particular part of the code looks suspicious? Then sprinkle it liberally with breakpoints or print statements (as your environment allows) and prove to yourself it's behaving properly. Check the input variables. Check the output variables. Hell, maybe even check the environment variables.

When I'm stumped, I try to fall back on basic experimentation: gather data, form hypothesis, test hypothesis, lather, rinse, repeat. You know, toss some "science" into my Computer Science. :-)

Answer 19

Forget the debugger. Waste of time, most of the time.

First, rubber duck it. If that fails, move to plan B.

Plan B: ensure you have good unit tests. Then you can home in on the problem (assuming you can reproduce it) using techniques such as what Kent Beck calls the "Saff squeeze".

Answer 20

Find someone else who understands the basic issues and technology (so they don't give you a blank stare) and explain the problem to them. They often don't even need to answer, just organizing your thoughts enough to explain the problem to someone else often brings the solution into focus.

EDIT: I was referring to Rubber Ducking even though I couldn't put a name to it. Thank you Morendil.

Answer 21

Know the value of what you're trying to fix. Don't waste time on "fluff". Document, work around and move on if value less than cost.

If value>cost, and you've had a good crack at it, then communicate it. Many times a tough problem is only a tough problem due to the perspective of the person involved. To others, things can often be highly obvious. Don't use your team as a crutch, and definitely respect their time etc but by the same token realize and tap the value of having workmates and peers.

Answer 22

Set a time limit on how long at once you will work on the task, e.g. 2 hours. Once this time is reached and you aren't making progress, take a break for 10-15 minutes which can be going for a walk, eating a snack, chatting with people, whatever besides that problem that can help "reset" your mental focus as after a while, it is useless to try to fix things through brute force.

Make sure you have a clear understanding of what the bug is exactly. There have been times where I'd get a bug that said, "This looks ugly. Please fix," and think, "Where is the spec that I'm supposed to make this look like as I don't know how to make this look good?" sort of thinking. This can also come if you've had to go to someone to referee the finger pointing between developers and testers where each is saying, "I am following the spec." That referee could be a manager, of developers, testers, or projects, but I do feel sorry for those that get pulled into these situations.