tags:

views:

55

answers:

1

Hi,

My friend asked me this question today. How to test a vending machine and tell me its test cases. I am able to give some test cases but those are some random thoughts. I want to know how to systematically test a product or a piece of software. There are lots of tests like unit testing, functional testing, integration testing, stress testing etc. But I would like to know how do I systematically test and think like a real tester ? Can someone please explain me how all these testings can be differentiated and which one can be applied in a real scenario. For example Test a file system.

+2  A: 

Even long-time, well respected, professional testers will tell you: It is an art more than a science.

My trick to designing new test cases starts with the various types of tests you mention, and it must include all those to be thorough, but I try to find a list of all the ways I can interact with the code/product.

For the vending machine example, there are tons of parts, inside and out.

Simple testing, as the product is designed to work, gives plenty of cases

  • Does it give the correct change
  • How fast can it process the request
  • What if an item is out of stock
  • What if it is overfilled
  • What if the change drawer is full
  • What if the items are too big, or badly racked
  • What if the user puts in too little money
  • What if it is out of change

Then there are the interesting cases, which normal users wouldn't think about.

  • What if you try to tip it over
  • Give it a fake coin
  • Steal from it
  • Put a coin in with a string
  • Give it funny amounts of change
  • Give it half-ripped bills
  • Pry it open with a crow-bar
  • Feed it bad power/brownout
  • Turn it off in the middle of various operations

The way to think like a tester is figure out every possible way you can attack it, from all the "funny cases" in usual scenarios, to all the methods that are completely outside of how it should be used. Any point of input, including ones you might think the developers/owners have control over, are fair game.

You can also use many automated test tools, such as pairwise test selection, model-based test toolkits, or for software, various stress/load and security tools.

Merlyn Morgan-Graham
@Merlyn Thank you very much for the great answer. But if I am given some 2000 lines of code like a file system or a web browser for example because file system requires lot of testing from black box to white box. how should I approach it. From your answer I should go with the interface like read write etc. But its more than that. how it works if multiple processes issue reads etc
brett
@brett: There are many good books on testing. You might want to start with one of those. As for "lines of code", you can look into tools like code coverage. My answer was more to get the wheels turning, rather than trying to be exhaustive. Trust me, it takes years to learn good testing...
Merlyn Morgan-Graham
@brett: Here is an interesting link for reviews of testing books: http://www.testingreflections.com/node/view/1310
Merlyn Morgan-Graham
@Merlyn can you recommend me any good books on testing. I am c/c++ and I like kernel programming.
brett
Thank you very much
brett
@brett: I don't know how much testing you have already done. Looking into and helping out on the Linux kernel's test suite may be an interesting exercise for you. Unit testing is fairly easy to get started with, compared to all the "we have millions of dollars in budget" testing :)
Merlyn Morgan-Graham
@brett: "But its more than that. how it works if multiple processes issue reads etc". Concurrency is one of many areas you must approach when testing software. Keep adding to your list of ways to test. Make sure you actually write up that list! If you try to keep it all in your head, you will forget some of it. It can be a high level list, but it will help you figure out how each piece fits in relation to the others.
Merlyn Morgan-Graham
@Merlyn Can you please tell me what test methodologies do I need to check when testing a file system (like unit testing, functional). Also now i read about model checking. How can I model check a file system ?
brett
@brett: Model based testing is a methodology/set of tools where you make a model of how you think the system should work (like UML/state machines, etc). You create code-generators for the individual parts of that system. The tools generate test cases and code for you. You can create models that are very high-level, very low-level, or both. To model a file system, you should first figure out exactly how that file system works, at the level you are trying to test. I can't get into those details for you, though, because I've never done model based testing myself :)
Merlyn Morgan-Graham
@brett: As for the rest of these questions, it seems you should read up and get a better understanding of the different categories of tests. Without spoiling too much for you: unit tests are usually functional tests. They aren't mutually-exclusive. Also, learn more about file systems. The key to testing isn't just about testing - it is more about writing good code and designing good programs.
Merlyn Morgan-Graham
@Merlyn Thank you very much. By the way which book from the above list do you ask me to start with
brett
@brett: I haven't read any of them, so I couldn't give you a good answer. The "top four" look like good places to start looking, and from there, you can check out reviews on Amazon. I make sure to check out a low-star review as well as a high-star review, to see what people complain about, and if it matters to me.
Merlyn Morgan-Graham
@Merlyn Thank you very much.
brett