how do you troubleshoot with "works on my machine" scenarios

Question

It happens lot of the time that when you report a bug to a developer, he comes back saying "it works on my system" though its a browser app. How do you go about sorting that out ?

Answer 1

Try to recreate the user that found the bug's system as much as possible: From server config to machine config including browser and OS and such. You should probably have several different setups on which to test your app before releasing.

Answer 2

From a training/process point of view:

Train your team to know that "works on my machine" is not a get-out-of-jail-free response.

Have an automated build server.

Have an automated test deployment.

Your developers must know that "works" is defined as "works on the test server", not just their machine.

From a testing/debugging point of view:

The developer needs to be shown the sequence of actions that result in the bug happening.

You might want to capture screenshots showing the bug, or possibly a video capture (using tools such as Camtasia). People can be quite bad at describing the sequence of actions that they performed on a system that led to a bug showing itself, so the more information you can capture about the bug and how to replicate itself the better.

From a development/environment point of view:

If there is genuinely a bug that exhibits itself on one environment but not the developer's then find out if it exhibits itself on no development environments, or just your one developer's.

From thereon in it is a case of trying to reduce the differences between the two environments so that your developer can see the issue on his machine.

Or you can go the other way and attempt to debug the issue on the production (non-development) environment.

Implementation details of these vary by platform.

Answer 3

We deal with that problem by having a development environment on top of the local development that is as close to the productive system as possible in terms of setup, hardware, etc. As a result almost all problems that occur in the production environment are reproducible on that development system even if they can't be reproduced on local developer machines.

Answer 4

IE Tester is a good tool for this kind of troubleshooting. If you need to test lots of browsers then virtual machines like Virtual PC are your best bet so you can have many client set-ups on your test server.

Answer 5

This is a common escapist retort that I encounter from teams. My response usually is: "You know, your system isn't the production server and that's where it needs to work". In other words, that excuse simply isn't acceptable.

I also indicate to them the possibilities:

a. There is a configuration difference between the local system and the server.

b. Certain dependencies of the functionality are not updated on the server.

c. They haven't cleared their browser cache.

d. I replicate the problem on the Staging server and demonstrate it to them.

e. ... and so on, depending on the case.

Answer 6

You need to give as much information to the developer as possible. Even stuff that you don't think is relevant.

I can't count the number of times I've had a problem reported and couldn't repeat it, only to find out later a piece of information that the user hadn't originally included but was the key to unlocking the puzzle.

You also need to not accept that answer and say "well something must be different between your set up and mine, what can we do to sort it out".

Answer 7

ahh yes... the oldest excuse in the book.

Assuming that both the developer and the tester is testing on the same server I would try to isolate the bug by identifying what the difference is between the developers machine and the testers machine. Could be a something minor like flash version, browser differences or forgetting to clear your browser cache

I would also recommend using an automated testing framework and test apps on a dedicated test server.

Answer 8

Not much you can do as an end user, but as a developer you can avoid a lot of these issues by including a lot of logging in the system - The differences the user will think of will just be the simple things you have tested already, but good logging lets you see exactly what was happening when the system failed. I've found quite a few bugs that couldn't possibly happen that way.