Web Automation Tool

Question

I've realized I need a full-fledged browser automation tool for testing user interactions with our JavaScript widget library. I was using qunit, starting with unit testing and then I unwisely started incorporating more and more functional tests. That was a bad idea: trying to simulate a lot of user actions with JavaScript. The timing issues have gotten out of control and have made the suite too brittle. Now I spend more time fixing the tests, then I do developing.

Is it possible to find a browser automation tool that works in: Windows XP: IE6,7,8, FF3 OSX: Safari, FF3 ?

• I've looked into SeleniumIDE and RC, but there seems to be some IE8 problems.

• I've also seen some things about Google's WebDriver, which confusingly seems to work with Selenium.

• Our organziation has licenses for IBM's Rational Functional Tester, but I don' think that will work on the MAC.

The idea is to try to run tests on all the browsers our organization supports. Doable? Are my requirements unrealistic? Any recommendations as far as software to try?

Thanks!

Answer 1

Selenium is probably your best bet out of the tools you mentioned. What are the issues it has with IE8? You might want to check out HttpUnit to see if that meets your needs, also.

Answer 2

I'm in the same boat. It is a difficult problem to solve. Windmill and Selenium are the 2 best I've found. Though they both have issues. Selenium can only record scripts in Firefox and I haven't managed to get the proxy chaining to work as advertised. Windmill you can record in any browser and you can supposedly tweak the proxy to put extra logic in there, but the js mechanism for recording across page loads has been in my experience very brittle at least on the app I have to test.

I don't think anyone can get it quite right as long as there is more than one browser that needs to be supported.

Answer 3

I would recommend using Selenium but I say that as a Selenium Committer.

Selenium works on any browser that supports JavaScript since the framework has been written in JavaScript. This means if your browser on any OS supports JavaScript it will run in Selenium. That documentation it out of date, you can see that since it is talking about IE8b1 and IE9 preview is out now.

Selenium and WebDriver (which isn't a Google thing since it started at ThoughtWorks) are currently being merged as they both have their strengths and weaknesses. The current merged work will be called Selenium 2 and you can start using the alpha release now at http://code.google.com/p/selenium/. It will still work on any OS as that is still the main driving force behind the work being done.

Selenium IDE only works on Firefox because it is a Firefox add on.

I personally would avoid Rational Functional Tester because it has a lot of weaknesses that its not even worth contemplating.

If you start with Selenium there are some tutorials on my site at http://www.theautomatedtester.co.uk

Answer 4

Maybe have a look at SIKULI. It's a different paradigm but, depending on what you want to test exactly, it may do the job and will work with any browser, on any platform. Have a look at their official blog for some examples of interactions with web applications.

Answer 5

Selenium RC is a great tool if you invest the time to use it. With significant modifications to the existing library I've gotten it to fulfill all of my front end testing needs.

The confusion you are having about Webdriver is understandable. Selenium 2 is in development and will be a merge of Webdriver and Selenium. Check out: http://www.youtube.com/watch?v=RQD4EzWI4qk to get more detail.

The only browser that I have found to be unusable with Selenium is IE6. IE7 and IE8 work fine as does Firefox (which I have modified to include firebug for debugging purposes).

Answer 6

So I wrote some of my more problematic tests in Selenium RC, using the Python driver. It was a better experience than writing the same tests in pure JavaScript, but I still had some of the same issues.

Testing something like an ajax autocomplete widget, meant forking some of the code depending on IE, or Firefox, and I still can't get typeKeys or a combination of type with typeKeys to work in Safari.

So, I am not sure if having cross browser clean, extensive ui tests is a bit unrealistic.

Should I try webdriver/Selenium 2? Would that make things better, or is that product not ready for prime time yet? How's the Python binding for that? I don't know Java, but I would learn some if need be.

Answer 7

Try Sahi (http://sahi.co.in/) It works across browsers and operating systems. It has a powerful recorder, and great APIs for object identification. It supports HTTPS, proxy tunneling etc. and has drivers in sahi script, java and ruby. It also has parallel playback inbuilt. It is 5 yr old mature project hosted on SourceForge, with releases almost every month.

It automatically waits for AJAX and page loads, and does not use XPaths for object identification. It also handles sites with dynamic ids.