views:

1891

answers:

16

Ok, maybe I'm missing something, but I really don't see the point of Selenium. What is the point of opening the browser using code, clicking buttons using code, and checking for text using code? I read the website and I see how in theory it would be good to automatically unit test your web applications, but in the end doesn't it just take much more time to write all this code rather than just clicking around and visually verifying things work?

I don't get it...

+8  A: 

If your application is even 50+ pages and you need to do frequent builds and test it against X number of major browsers it makes a lot of sense.

zodeus
Yes but looking at the example on the site...verifying text appears, isn't something that would need to be done programatically. How often do you change 50 pages at once?
Greg
Every time that you change a library file that holds utility functions used on all 50 pages.
EBGreen
Or in our app its actually 200+ pages. It would take days for a human to do a full regression.
zodeus
It would probably take 100 days I'd think to write out a full unit test for stuff you just see as intitive. Like on this page how many links are there? 100? Would you write a test case for everything?
Greg
It all depends on what you want to do.
zodeus
@Greg: You would usually write these tests according to your use cases. Imagine a user's moves on the site: Reading the list of latest questions should show them; having used a sample database there should be a difference when adding a new question. That question shouldn't be double-escaped etc.
Energiequant
@Greg: you go gradual about it, adding tests for the specific features/changes you are working on. If you are using the page object pattern, those will help you when you want to test a bigger cycle of interactions. I updated my answer with more info, check it out.
eglasius
@Greg, it takes time to write the tests yes, but they run much faster than a human can. Right now I'm working with tests that take ~12 hours to run but it would take a human several days to manually run the equivalent tests. Take that time savings and multiply it over several years, that a lot of time!
Wesley Wiser
+15  A: 

This is a common thing that is said about unit testing in general. "I need to write twice as much code for testing?" The same principles apply here. The payoff is the ability to change your code and know that you aren't breaking anything.

EBGreen
+1 for piece of mind when making changes :)
eglasius
That's the entire point of unit testing, for me
matt b
+1  A: 

And if you save those tests as JUnit classes you can rerun them at your leisure, as part of your automated build, or in a poor man's load test using JMeter.

duffymo
+25  A: 

It allows you to write functional tests in your "unit" testing framework (the issue is the naming of the later).

When you are testing your application through the browser you are usually testing the system fully integrated. Consider you already have to test your changes before committing them (smoke tests), you don't want to test it manually over and over.

Something really nice, is that you can automate your smoke tests, and QA can augment those. Pretty effective, as it reduces duplication of efforts and gets the whole team closer.

Ps as any practice that you are using the first time it has a learning curve, so it usually takes longer the first times. I also suggest you look at the Page Object pattern, it helps on keeping the tests clean (http://code.google.com/p/webdriver/wiki/PageObjects).

Update 1: Notice that the tests will also run javascript on the pages, which helps testing highly dynamic pages. Also note that you can run it with different browsers, so you can check cross-browser issues(at least on the functional side, as you still need to check the visual).

Also note that as the amount of pages covered by tests builds up, you can create tests with complete cycles of interactions quickly. Using the Page Object pattern they look like:

   LastPage aPage = somePage
      .SomeAction()
      .AnotherActionWithParams("somevalue")
      //... other actions
      .AnotherOneThatKeepsYouOnthePage(); 
  // add some asserts using methods that give you info
  // on LastPage (or that check the info is there).
  // you can of course break the statements to add additional 
  // asserts on the multi-steps story.

It is important to understand that you go gradual about this. If it is an already built system, you add tests for features/changes you are working on. Adding more and more coverage along the way. Going manual instead, usually hides what you missed to test, so if you made a change that affects every single page and you will check a subset (as time doesn't allows), you know which ones you actually tested and QA can work from there (hopefully by adding even more tests).

eglasius
A: 

The point is the ability to automate what was before a manual and time consuming test. Yes, it takes time to write the tests, but once written, they can be run as often as the team wishes. Each time they are run, they are verifying that behavior of the web application is consistent. Selenium is not a perfect product, but it is very good at automating realistic user interaction with a browser.

Benjamin Lee
+18  A: 

Don't worry, when you have to test it your self, you'll see the point of automation.

OscarRyz
I already do, I can't even think up any test cases though, and the website gives pointless examples like verifying text appears. My web app is much more complicated than that.
Greg
Start small, make tests for tiny things - lots of small tests add up to a large amount of coverage over time
matt b
@Greg - It's not just verifying text on a page, it's verifying the interactions that cause specific text to appear or behaviour to happen. For example, "If a user does X on a page 1 then they should see the message Y on page 2.".
Hates_
+1  A: 

In a past job we used to unit test our web-app. If the web-app changes its look the tests don't need to be re-written. Record-and-replay type tests would all need to be re-done.

Ben S
+2  A: 

The point is the same as for any kind of automated testing: writing the code may take more time than "just clicking around and visually verifying things work", maybe 10 or even 50 times more.

But any nontrivial application will have to be tested far more than 50 times eventually, and manual tests are an annoying chore that will likely be omitted or done shoddily under pressure, which results in bugs remaining undiscovered until just bfore (or after) important deadlines, which results in stressful all-night coding sessions or even outright monetary loss due to contract penalties.

Michael Borgwardt
From my experience, with selenium or similar frameworks, the relation of effort isn't that bad. Specially if compared to actually verifying it /completely/ by hand. Now, that relation usually comes when comparing adding to an already coded system where tests are usually missing plenty of features.
eglasius
A: 

If you do not like the Selenium approach, you can try HtmlUnit, I find it more useful and easy to integrate into existing unit tests.

Jonas Klemming
+5  A: 

Imagine you have 50 pages, all with 10 links each, and some with multi-stage forms that require you to go through the forms, putting in about 100 different sets of information to verify that they work properly with all credit card numbers, all addresses in all countries, etc.

That's virtually impossible to test manually. It becomes so prone to human error that you can't guarantee the testing was done right, never mind what the testing proved about the thing being tested.

Moreover, if you follow a modern development model, with many developers all working on the same site in a disconnected, distributed fashion (some working on the site from their laptop while on a plane, for instance), then the human testers won't even be able to access it, much less have the patience to re-test every time a single developer tries something new.

On any decent size of website, tests HAVE to be automated.

Lee B
+1  A: 

Why do you need Selenium? Because testers are human beings. They go home every day, can't always work weekends, take sickies, take public holidays, go on vacation every now and then, get bored doing repetitive tasks and can't always rely on them being around when you need them.

I'm not saying you should get rid of testers, but an automated UI testing tool complements system testers.

+11  A: 

Because you can repeat the SAME test over and over again.

Luke
+2  A: 

Selenium (along with similar tools, like Watir) lets you run tests against the user interface of your Web app in ways that computers are good at: thousands of times overnight, or within seconds after every source checkin. (Note that there are plenty of other UI testing pieces that humans are much better at, such as noticing that some odd thing not directly related to the test is amiss.)

There are other ways to involve the whole stack of your app by looking at the generated HTML rather than launching a browser to render it, such as Webrat and Mechanize. Most of these don't have a way to interact with JavaScript-heavy UIs; Selenium has you somewhat covered here.

undees
+2  A: 

Selenium will record and re-run all of the manual clicking and typing you do to test your web application. Over and over.

Over time studies of myself have shown me that I tend to do fewer tests and start skipping some, or forgetting about them.

Selenium will instead take each test, run it, if it doesn't return what you expect it, it can let you know.

There is an upfront cost of time to record all these tests. I would recommend it like unit tests -- if you don't have it already, start using it with the most complex, touchy, or most updated parts of your code.

Jas Panesar
A: 

For applications with rich web interfaces (like many GWT projects) Selenium/Windmill/WebDriver/etc is the way to create acceptance tests. In case of GWT/GXT, the final user interface code is in JavaScript so creating acceptance tests using normal junit test cases is basically out of question. With Selenium you can create test scenarios matching real user actions and expected results.

Based on my experience with Selenium it can reveal bugs in the application logic and user interface (in case your test cases are well written). Dealing with AJAX front ends requires some extra effort but it is still feasible.

Petteri Hietavirta
A: 

I use it to test multi page forms as this takes the burden out of typing the same thing over and over again. And having the ability to check if certain elements are present is great. Again, using the form as an example your final selenium test could check if something like say "Thanks Mr. Rogers for ordering..." appears at the end of the ordering process.

Polygraf