views:

534

answers:

19

We're neck deep in a project right now, schedules are tight (but reasonable). Our general strategy is to get a strong beta done, release it for testing, and get feedback from our testers.

Quite frequently, we're being hit by small things that spiral into long, time-costing discussions. They all boil down to one thing: While we know what features we need, we are having trouble with the little details, things like 'where should this message go' and 'do they need this feedback immediately, or will it break their flow, so we should hold off'?

These are all things that our testers SHOULD catch, but

a) Each 'low priority' bug like this drains time from critical issues

b) We want to have as strong a product as possible

and

c) Even the best testing group will miss things from time to time.

We use our product, and we know how our users use the old version...but we're all at a loss as to how to think like a user when we try to use the new version (which has significant graphical as well as underlying changes).

edit - a bit more background:

We're writing a web app used by a widely-distributed base of users. Our app is a big part of their jobs, but not the biggest (and, of course, we only matter to them when it doesn't work). Getting actual users in to use our product is difficult, as we're geographically distant from the nearest location that serves as an end user (We're in Ohio, and I think the nearest location we serve is 3+ hours away).

The closest we can get is our Customer Service team (who have been a big help, really) but they don't really think like the users either. They also serve as our testers (it really motivates them to find bugs when they know that any they DON'T find may mean a big upswing in number of calls). We've had three (of about 12 total) customer service reps back here most of the week doing some preliminary testing...they've gotten involved in the discussions as well.

+8  A: 

Speaking generally, you can't. There's not any way you can turn off the "programmer" part of your brain and think like a user.

And you're right about (c), testing groups don't necessarily catch all the bugs. But the best thing you can do is get a testing group comprised of real, honest-to-goodness end users, and value their feedback. Draw further conclusions from their general comments.

If you want to know how your users will see your system, the closest you can get is usability testing with real users. Everything else is just heuristics and experience, and is also subject to error. There's no such thing as a bug-free product, but you should be able to get a "strong" product with usability testing.

Adam Bellaire
partial frontal lobotomy? ;)
Andrew Rollings
I strongly disagree with this. Perhaps your typical user isn't a CS grad, but they generally know their own job very well. In that realm they are the genius and you need to play "grasshopper" at their feet for a while.
T.E.D.
@ted: That's definitely true in some cases, but I maintain you still can't learn to think like them. You'll still be a programmer with domain-specific knowledge that makes you think about the app differently from them. Also, the OP talks about a web app for a wide base of users, not specialists.
Adam Bellaire
@ted: My point was not to disparage the intelligence of capabilities of users, but to point out that as programmers, we are "handicapped" when it comes to thinking like them.
Adam Bellaire
+5  A: 

Buy a cheap, easy to use video camera and record your testers using the app. Even better, get some people unfamiliar with the app. to use it and video them. It's relatively cheap, and you'd be surprised what it will highlight.

Mitch Wheat
+1  A: 

The flippant (yet somewhat accurate) answer to how to think like a user is put a knitting needle in your ear and push really hard.

The longer response is that we as programmers are not normal and I mean that in a good way. I scratch my head at the number of people who still run executables they receive from strangers in emails and then wonder how their computer got infected.

Any group of people will in time develop their own jargon, conventions, practices and expectations. As a programmer you will expect different things from an operating system than Joe User will. This is natural, to be expected yet hard to work around.

It's also why BAs (business analysts) exist. They typically come from a business or testing background and don't think like programmers. They are your link to the users.

Really though, you should be talking to your users. There's no poitn debating what users do. Just drag a few in and see what they do.

cletus
+10  A: 

Read Don't make me think.

Dead account
A: 

The 'right' way to do this is to prototype (or mock up) your new interface features, and watch your users try to use them. Nothing is as enlightening as seeing a real user try to use a new feature.

Unfortunately, given most projects time and resources, this is not possible. If that is the position you are in I would recommend you discuss in the team who has the best grasp of usability, and then make them responsible for usability decisions - but that person will need to regularly consult real users to make sure his/her ideas are consistent with what the users want.

DanSingerman
A: 

I'd suggest doing some form of usability testing; I've participated in such in the past, and found them quite useful.

If you were writing a ticketing system, for example, bring up tasks, and ask questions like "how would you update this ticket" or "what do you expect to happen if this button is clicked".

You don't necessarily need a full application, either, in some places screen shots can be used.

warren
A: 

You could take the TDD/BDD approach and get the users involved before beta, having them work with you on refining requirements as you write your unit tests. We're beginning to incorporate some of those trends into our current project, and we're seeing fewer bugs in the areas where we have involved the users earlier.

Ryan Riley
+16  A: 

Watching someone using the app is a huge benefit to me. Possibly someone who is not entirely familiar with it.

Seeing how they try to navigate, how they try to enter information or size windows. Things we take for granted after creating/running the app hour after hour, day after day.

Users will always try and do things you never expected and watching them in action might bring to light how you can change something that might have seemed minor, but really makes a big impact on them.

Dustin Brooks
Agreed. However, I typically find it more informative to watch an "expert" user (or at least a familiar user). If they are running through weird circumloqutions without thinking about it to do normal things, you know where to start your redesign.
T.E.D.
yep! and most important of all.. DO NOT INTERFERE :) don't teach/train your testing subjects!
Ric Tokyo
the use of a video cam minimises coaching....
Mitch Wheat
A comment below recommended Don't Make Me Think, which is a short, clear and insightful book on precisely this kind of lightweight end-user testing.
glenatron
+1  A: 

A usability test group will help.. tests not focused on discovering bugs, but on the learning curve of the new design, made by a group of users, not programmers.

kajyr
Good general advice, I think, and definitely something we'd do, if we could. Sadly impractical in the current situation.
Jeff
+3  A: 

I like policy of "eating your own dog food"("http://en.wikipedia.org/wiki/Eat_one's_own_dog_food). It brings you one step closer, because you become a user, although you might think like one.

Robert Gould
This doesn't really work for true testing, as if you're a developer, you know all the little quirks that a new user would not be used to. You'll subconsciously avoid features that may not work as well, and you won't do the unexpected things users really would.
Andrei Krotkov
that's why I said "one step closer"
Robert Gould
+3  A: 

The important thing is to get enough information that you yourself can become a "user". Once you do that you can answer most questions yourself.

The way I always do this is to go talk with them about what they need to do, what they typically do, and how they use their current tools to do it. Then (very important) sit with them while they do it. Make sure you get on with them well enough that you can come back to them with questions about how they handle edge cases you think of later (often the answer will be the appalling "we go around the system manually for that").

I will almost always notice something they are doing that is a royal PITA that they didn't bring up because they are used to having to do that and don't know any better. I will always notice that their %90 typical workflow isn't the easiest workflow the tools provide.

You can't really rely on plain old-fashioned requirements gathering by itself, because that is asking them to think like a developer. They generally don't know what is possible to do with your software, what is easy, and what is hard. Also they typically have no clue on GUI design principles. If you ask them for design input they will just tell you to put any new control on their favorite page, until the thing looks like a 747 control panel.

T.E.D.
A: 

There is no "think like a user" technique, get your hands on someone who knows nothing of the project and throw what you have done at them.

It's the only way to see how the look + feel + functionality present themselves to the end user.

Once you shocked that person who knew nothing of the product, listen to all of their idiotic (or so you think they are) complaints, fix them, arrange every silly cosmetic thing they point out (either by fixing the UI or by improving whichever documentation you had)..

and after you have satisfied the person you chose to look at your app from zero knowledge on the subject first round, pick another ...and another... until they stop being shocked when they see it, and they don't get stuck on.. "ok.. what does this do?" kind of phases.

You (as a member of the project, be it the project manager, developer, etc) will never think like a user is my answer to that question.

Ric Tokyo
+3  A: 

Try to use your app when you are very hurry (e.g. you have someone who waits for a dinner). You will see all this little things because you have to wait, you have to go back to the mouse of the keyboard, etc.

And also, make your wife use it. Or your mother.

Another useful test : help someone to use it, by phone. If he can't find the button with your directions, that's probably a bug.

ofaurax
A: 

I treat all users like malicious idiots.

Malicious because I assume all users are going to try and break my code, do stuff that is not allowed, avoid typing in valid data, and will do anything in their power to make my life hell.

Idiots because again I can't assume they will understand simple stuff like phone formats, will run away screaming if presented to many choices, and will not make any leap of faith on complicated instructions. The goal is to hold their hand the entire way.

At the same time, its important to make sure the user doesn't realize you think they're an idiot.

David
+1  A: 

To think like a user, be one. But are these actually bugs that your testers are reporting? Or are they "enhancement requests"? If the software behaves as designed per requirements and they just don't like the way it operates, that's not a bug. That's a failure of requirements and design. Make it work, make it rock solid, make it easy to change and you'll be able to make it what your users want.

Rob K
A: 

Old saying: You can make something "fool proof" but you can't make it "Damn-fool proof".

Additionally: When you make something "idiot proof" the world invents a better idiot.

Other than that, I agree with what everyone else said.

Brian Postow
+2  A: 

The problem is often that even the users don't know what they want until they are actually working with the software. Sometimes, a small oversight can be a big usability problem, sometimes a well thought out function that was requested by many users sees only little use.

My suggestions to decrease the risk of not implementing the right usability features:

  • Take a look at users actually doing their day to day work. Even if they use another software or no software at all. You will be able to determine the artifacts they often need to get their job done. You will see what data they frequently need. Concentrate on the artifacts, data and workflows most used. They should be the most usable. Exotic workflows may be a bit more time consuming for the users than often used workflows.

  • Use working prototypes of the GUI to let users work through a realistic workflow. Watch them and note what hinders them and what works well. Adjust your prototypes accordingly.

  • If an issue arises in an often-used part of your software, it is time to discuss it now and in details. If the issue concerns a seldom used part, make it a low priority issue and discuss it if you have the time. If issues or suggestions are low priority, they should stay low priority. If you can't determine if solution A or solution B is the best, don't run in circles with the same arguments over and over. Just implement one of the solutions and see if the beta testers like it. The worst thing you could do is waste time over tiny issues, while big issues need to be fixed.

  • A software will never be perfect, because the viewpoints of users differ. Some users will think that a minor problem breaks the whole application. Others will live with even severe usability issues. People tend to lend their ear to those who argue the loudest. Get to know your users to separate the "loud" issues from the important ones. It takes experience to do this, and sometimes you will make wrong decisions, but there is no perfect way, only one of steady improvement.

  • If you can, set aside a certain amount of usability development resources for the rollout phase of your software. Usability issues will arise when people start working with it in a real production environment. Sometimes it is not important to present the perfect software, but to solve issues quickly as they arise.

Sebastian Dietz
A: 

Ask someone with absolutely no knowledge, insight or programming experience to use the program and try to figure out every function of the program.

People who would NEVER use such a program are most likely to find bugs.

See it as a new Safari user (or FF) who tries to put the URL inside the search field... As a programmer you guess no-one would be that stupid (or, well.. unknowing), but people actually sometimes find themselves in these situations. As a programmer, we miss these things.

xaddict
+1  A: 

I see some good suggestions here, especially observing people trying to use you app. One thing I would suggest is to look at the order in which things are presented to the user on paper forms (if they use these to do data entry from) and make the final data entry page mimic that order as closely as possible. So many data entry errors (and loss of data entry speed) are from them having to jump around on the page and losing their place. I did some work for a political campaign this year and in every case, entering data was made much more difficult because the computer screen did things in a differnt order than the paper inputs. This is particularly important if the form is one that can't be changed (like a voter registration form, a campaign has to use what the state provides) to match the computer screen. ALso be consistent from screen to screen if possible. If it is first Name last name on one form, making it last name first name on the next will confuse people and guanteee data entry errors.

If you are truly interested in understanding users though I strongly suggest taking a course in Human factors engineering. It is an enlightening experience.

HLGEM