There's no denying the benefits of automated testing. Automation increases test reliability and speed. With technologies like xUnit and Fit is it possible to forgo manual testing altogether?
The one weakness of these frameworks seems to be the UI but there are free and commercial UI automation testing suites as well.
Is manual testing no longer necessary?
Of course even if all tests can be automated analyzing a failed test is still a manual process.
I highly doubt it. Beyond pass/fail, there's ease of use, look & feel, and things of that nature. That kind of stuff can only be "tested" by humans.
Doubtful,
Testing also allow you to find inconsistencies in the software that produce correct result, e.g. software not being intuitive or plain unusable.
Also there can be bugs in the testing method which needs to be tested, so there will always be a level of testing required.
Automation is extremely useful and you should have as much of it as you can.
However, it should not replace your human integration tests. A user interface is to be used by humans, and so there should be tests performed by humans.
No, because automated testing only covers cases of those that developed the tests.
Human testing (hallway testing) should still be performed, and cannot be automated.
Joel lectured/wrote on this:
http://www.joelonsoftware.com/backIssues-2007-12.html
"This is how you get unit testing, which is not a bad thing, don’t get me wrong, and it’s how you get all these attempts to mechanically “prove” that a program is “correct.” The trouble is that anything that can’t be automated has to be thrown out of the definition of quality. Even though we know that users prefer software that looks cooler, there’s no automated way to measure how cool looking a program is, so that gets left out of the automated QA process."
"So, the bottom line is that if there really were a mechanical way to prove things about the correctness of a program, all you’d be able to prove is whether that program is identical to some other program that must contain the same amount of entropy as the first program, otherwise some of the behaviors are going to be undefined, and thus unproven."
Yes, at a minimum, if humans are going to use your application, you need to have humans testing it to make sure they can actually use it correctly.
To add to what others have said on the "Automated testing will never replace people testing"
Joel wrote about the this specifically about how an imbalance between automated & people testing damaged Windows Vista in one of his talks at yale
The old testers at Microsoft checked lots of things: they checked if fonts were consistent and legible, they checked that the location of controls on dialog boxes was reasonable and neatly aligned, they checked whether the screen flickered when you did things, they looked at how the UI flowed, they considered how easy the software was to use, how consistent the wording was, they worried about performance, they checked the spelling and grammar of all the error messages, and they spent a lot of time making sure that the user interface was consistent from one part of the product to another, because a consistent user interface is easier to use than an inconsistent one.
None of those things could be checked by automated scripts. And so one result of the new emphasis on automated testing was that the Vista release of Windows was extremely inconsistent and unpolished. Lots of obvious problems got through in the final product… none of which was a “bug” by the definition of the automated scripts, but every one of which contributed to the general feeling that Vista was a downgrade from XP. The geeky definition of quality won out over the suit’s definition; I’m sure the automated scripts for Windows Vista are running at 100% success right now at Microsoft, but it doesn’t help when just about every tech reviewer is advising people to stick with XP for as long as humanly possible. It turns out that nobody wrote the automated test to check if Vista provided users with a compelling reason to upgrade from XP.
Considering the huge improvements that have been made in this field these last years, I have no doubt that someday it will be the case. But certainly not in the near future.
It depends on the purpose of your testing is.
Automated testing works well enough for checking if the code works, but how would you imagine using automated testing for a program like Photoshop?
The automated test can check if your crop function works, but it won't tell you if it's easy to use or if users can find it at all.
Perhaps you can give an example of what you use it for?
I'm going to argue the opposite for a moment, since the question and the answers so far come from application engineers who think in terms of human interfaces.
If you're testing embedded software, where inputs and outputs are electronic and often operate many times faster than human operators, manual testing is incredibly difficult. In such cases any test that you run is run using a script, or some sort of programmatic method. Therefore, in those cases, completely automated testing is a very reasonable approach.
Of course, the definition is blurred - every automated test starts out as a manual test until it's added to the test suite, if you like to think of it that way.
For user interfaces, fine. Programs can't tell when your CSS has gone awry and your page is simply ugly without being so restrictive that they break with every change. But not everything is a web application or a GUI.
As many unit tests as one may build into their code, there is not yet, and may never be, a comprehensive way to automate integration testing. Also for stand alone applications it is still a large part of the testing process to install and use the application on any supported hardware and OS.
I think the Joel on Software comments sums it up - No, Automation can not completely replace manual testers.
UI based automation tools can reduce the repetition a manual testers does, and allow them to broaden their testing scope, and the automation will provide faster results on defects, which allows us coders to fix a defect with less time spent reviewing the code to remember it's initial intent. So, they're good, but not human-replacing good... yet.