Out of these 3, which have you or has your company chosen to work with? Pros & Cons please. I'll be comparing them myself as well, but I'd like to hear what others have to say.
Also, please state which you have all tried (so that I know you have a good comparison of the 3).
I haven't looked into Sahi, but I can tell you why we have recently chosen Selenium RC over watir. It all comes down to programming language. We are a Java shop, and no matter how easy or great Ruby is, it is much easier to coax developers into writing tests if they don't have to change contexts or IDEs. We found functionality to be similar and had a preference for watir's object model, but being able to write Selenium RC tests in Java gave it the edge.
If you are looking at open-source web testing, you should also consider webdriver. It is a good alternative to Selenium RC, and is actually going to become Selenium 2.0 at some point in the future.
Selenium here but we haven't tried alternatives. Previous experience and quite ok ajax support were main reasons to use it.
I made a presentation for a seminar I had to hold for our R&D team. Here is basically what I came up with:
Pros:
Cons:
Pros:
Cons:
Pros:
Cons:
There are a lot of other small stuff that I didn't include.
I decided to use Selenium because I like the freedom that you get with all the languages it supports and because development is usually quicker because of its IDE recorder.
edit: I have done some of the same tasks in Watir and they seem to be more stable and straightforward in Watir (written in Ruby) than what I wrote in Selenium (in Java). Selenium has a lot of timing problems but Watir automatically waits for the page to finish loading, and can detect when it's finished loading. Whereas with Selenium, you have to use the pause command (or Thread.sleep() if you use it in Java).
automated testing is growing so large that it's a headache for me: selenium rc, watir, sahi, jameleon, junit, testng, ... and for me just picking one isn't enough and often have to use more than one... one of my main decision in picking a too is one that has capability to drive a web browser, to me that is the most important in testing web app.
i've tried jameleon in the past, as it a has jiffie plugin for driving IE as the browser but test case generation is often slow since there is not recorder/playback facility. feature that i liked best about jameleon is the jiffie snapshot which can take a screen shot and save it for later viewing if needed. i've stopped using jameleon since the jiffie plugin isn't suitable for running extended/repeated/stress test (don't recall but some problem where it crash after few iterations.
selenium rc + selenium ide (firefox plugin) for recording and playback of test case is great for generating test case on the fly. But I just ran into problem today with selenium rc with Internet Explorers with IE6 and IE7 test script runs fine in firefox but had different problem on PC with IE6 and IE7. This sucks since I need to test the web app with different browsers.
i haven't tried watir or sahi but agree with Dennis: with watir need to learn ruby and sahi interface is confusing (see their demo vid).
junit, i didn't try since it doesn't have plugin to drive any browser.
just my opinions...hope i didn't make any error regarding any tools; please do correct me if i'm wrong.
I represent a consulting company where one of our services include enabling manual software testers in automating testing activities. We have wide experience in software test automation using both commercial and open source tools.
Selenuim is a very popular, cross platform, cross browser tool offering choice of programming languages. But our experience tells us that the learning curve is on the higher side. It takes some time to understand the various tool sets of Selenium. For example, there is Selenium IDE, RC, Grid. What is the relationship among these tools? Where to use these tools. Many times I have heard people saying that Selenium has an IDE. But once we dig deeper, we realise that the IDE is nothing but a FireFox plugin...and it works only on FireFox. That means, Selenium allows us to record and play back only on FireFox and only if you install the plugin. What if i want to record using any other browser? This boils down to the fact that most of the points that are listed in the Pros of Selenium by the original author is valid ONLY if we use FireFox!!! Choice of programming language is a very powerful option, but then the automation engineer needs to have reasonable experience on the chosen language. Once we choose go with a programming language (other than Selenese), it is our responsibility to implement most of the things including logging mechanisms.
Because of the above mentioned reasons, we found it difficult to enable the testing team using Selenium. Recently we came to know about Sahi and our short experience with Sahi looks very promising. Let me discuss some of the points that our customers found interesting in Sahi.
I do agree that Sahi, despite being well developed, constantly improved and well supported, the community seems to be small as the rightly author has rightly stated. But that did not stop us from moving to Sahi and the feedback from our customers confirms that we are on the right track.
There is a customer testimonials area on Sahi's website. I have posted another experience of mine over here.
Hope this helps all those who are looking for a software test automation tool for automating their web site test automation
Cheers dp
I'm using Sahi and I think it very powerfull tool for functional testing. My opinions follows:
Pros:
Cons:
In conclusions a good choice for me ^_^ thanks Narayan
Roby
P.S: I tested also watin, selenium (integrated into Visual Studion using C# library).
I have to agree with Dennis that the pro-Sahi comments are probably from the Sahi team or some other interested party. When we tested Sahi in our shop a few months ago, it performed very poorly in comparison to Selenium. I don't know if it has improved, but at that point in time it was definitely not a usable product.
I have introduced Selenium to my company and implemented it pretty good between our DEV and TEST flows.
With all the features provided by selenium such as "wait for..." it's great for testing ajax applications.
We use JAVA through CruiseControl, so it is a simple matter converting the selenium test code to JAVA and use the code directly in our build scripts.
Selenium gets my vote any day!
Just to add another to the realm: We use iMacros because of its good Flash/Ajax testing support: http://wiki.imacros.net/Selenium
One problem I have found with Selenium is that it is very difficult to test complex multi-site apps. My company has an app that uses a login process via Facebook (using the Facebook API). In Selenium 1.0, which uses Javascript as its enabling engine, the single domain policy is enforced so that when I open my main site (http://beta.mysite.com) and then click on the Connect to Facebook button which opens a window in the www.facebook.com domain for logging in, my Selenium process loses the connection between the two windows and the app locks up at login. I have tried many different approaches to solve this and have not come up with an answer.
I switched to Watir because it provides an "attach" method which lets me programmatically access both the original domain window and the facebook window from the same test.
Also, as noted by several people, Selenium has timing issues. It is possible to use clickAndWait if a click is known to load a new page, but in our app, there are situations where clicking MAY cause an Ajax call and a dynamic in-page change in which case you do NOT want to clickAndWait (because your test app will hang waiting for a page to load that will never load) or it may cause a page load/reload, in which case NOT waiting is suicide because your app will simply run past the situation and start doing things before the page is fully loaded and ready to go.
If you can't tell which is going to happen (and in our case, you can't without introspecting the javascript which is MUY DIFICIL), you are up the proverbial creek without a paddle.
(Yes, I know, who in the hell would write javascript that does one or the other alternatively depending on certain contexts? I certainly wouldn't have written it that way and I will not defend the implementation of the app, I just have to figure out how to test it LOL!)
Anyway, this is also relatively easy to handle in Watir since you just let it decide whether to wait or not. To be fair, Selenium 2.0 (using WebDriver) will fix some of these problems, but unfortunately, the API will then be different. So its not an easy thing to figure out.
BTW, I was "driving" the Selenium API with Python and it took me about 2 hours to learn enough Ruby to be dangerous LOL and also to use Watir. I have a background in Smalltalk and Objective-C and Ruby comes quite second-naturedly (as did Python). In fact, I expect I am going to end up on the fence, because there are some awesomely nice, robust and symmetric things in Ruby that are much better than Python, but Python also has its pluses. I still think code is cleaner in Python and easier to read for the most part, but as for power (particularly in terms of metaprogramming), I think Ruby is the better engine.
Jon
i use sahi because its not using x path ,very simple and powerful. very active community.thanks narayan