Flex automated acceptance testing tools

views:

384

answers:

+2 Q:

Flex automated acceptance testing tools

I am looking for recommendations for tools for automated testing of a web application with some flex components.

To provide some background we have a web application that was entirely developed in AJAX+HTML and we were somewhat successful in using Selenium for testing that application end to end. We recently added some flex components into the mix and it got complicated.

We tried using Selenium Flex but we are disappointed with what it can do. So now we are looking for some alternatives. Ideally the tool would be able to drive both the web and the flex parts simultaneously, but we can also settle for just testing the flex components on their own. We prefer open source but good commercial tool is also an option.

I've heard about Fluint and FlexMonkey but haven't tried them yet. Anybody has any experience with using those? Any advice for someone who is just starting? Any gotchas?

Are there some other tools that could be used?

Thanks.

+1 A:

We've had some success using Fluint for unit testing and Quick Test Professional (with the Flex plug-in) for functional end-to-end testing. QTP is pretty heavy-weight, but once you've got it set up and have created a few tests, it works pretty well. It works with Flex as well as vanilla HTML/Javascript, so even if your app is a mix of the two, it should be able to handle it.

Fluint works great for unit testing, but given the amount of code required to test a component, I wouldn't advocate it for functional testing (it's too low-level). FlexMonkey is more of an automation framework for Flex Unit. You basically use it to record test cases, and it generates code using Flex Unit to perform the test. I've tinkered with it, but ultimately decided to use QTP.

As for tips to using Fluint: Learn how to use Sequences if you want to test a Flex component - it will make your life considerably easier.

Erich Douglass 2010-01-11 15:44:54

I've tried Fluint and it's awesome for unit testing components and interactions but it requires internal knowledge of application (white box testing) so it's not suitable for acceptance testing. QTP is right now not an option due to budget constraints (5k for single license isn't it?) but who knows if there is no alternatives I might look into it as well.

Gregory Mostizky 2010-02-04 06:56:41

+2 A:

i'v try FlexMonkey.

It was great, until he get lost in the recording of interaction with huge and complex Flex Component.

For me, and for now, it's great in the demo or with small app, but not ready for production.

BUT : seems to evolve quickly.

Antoine Claval 2010-01-12 10:51:39

After testing FlexMonkey for some time I completely agree with you. The potential is definitely there but current implementation is just lacking. I've had encountered some very basic bugs that should not have been in the 1.0RC version - such as text fields being cleared when adding some characters into them, controls reporting different values on consecutive runs and so on. Will probably revisit it in a year or so but for now it's just not good enough.

Gregory Mostizky 2010-02-04 06:52:59

+1 A:

I've had much success using FunFX to automate a fairly complex Flex application. FunFX is built off FireWatir, which is another web automation framework similar to Selenium. So together that should cover the entire AJAX/HTML + Flex bits of your application.

Since both FunFX and FireWatir are Ruby-based, I also recommend something like Cucumber as your testing framework to glue everything together.

James Bobowski 2010-02-03 04:14:41

I've seen FunFx mentioned couple of times on the web. I've looked at their website and had a distinct impression that the technology was not very mature. Pre 1.0 release, poor documentation, not updated very often (not sure about this one).What's your your assessment of it's maturity? Is it easy for someone who is not a contributor to get in and start working with it? Were there show stopper bugs that required fixing the code base? Thanks.

Gregory Mostizky 2010-02-04 06:49:26

One more followup question - what environment did you use for testing? Windows/Linux/Mac? And which browser?

Gregory Mostizky 2010-02-04 06:54:04

I had the unpleasant task of trying several of the Flex automation frameworks mid last year. None of them, at that time, were very mature. FunFX, right now, strongly reminds me of where Selenium was in its pre-1.0 days. It works most of the time, there are a few "missing features" ( some of which you will never see due to Flex automation limitations ), you have to explicitly code to catch problems ( its very asynchronous ), events are sent out and you have to manually check whether your app is now in the desired state.

James Bobowski 2010-02-05 02:01:00

The documentation is weak, but most likely because once you get an understand of how it works its very simple. I often have the elements.rb file open so I can find the names of all the components and what methods are available.There really aren't any severe bugs, however, I don't use the 0.2.2 gem. I pulled a copy from github, built and installed a gem from that, which I guess would be "0.2.3". I found it to work much better.I know it sounds like a pain, but Flex testing is a pain. :)

James Bobowski 2010-02-05 02:01:36

I have successfully used it on Windows Vista, Mac OS X, Ubuntu Linux and am tasked with moving my test environment to Windows 7 Enterprise within the month. The only major issue I've found is that last year, the JSSH plugin ( which you need for FireWatir ) somehow broke. I could only get it to work with an older-ish release of Firefox ( 3.0.8 ). This most likely has been fixed, since its been almost a year now. But I don't update my test VMs all too often, so I don't know for sure.

James Bobowski 2010-02-05 02:02:31

I was planning on writing several blog entries about how to use FunFX in real world testing, as well as how to code Flex applications to make automation easier. I suppose there is a demand for such things after all. :)ETA on those would be the second half of February at the earliest.

James Bobowski 2010-02-05 02:04:36

Also checkout Melomel. It's a simple library for interacting between Ruby and Flex. It has full Cucumber support baked right in too. The setup is pretty simple and there is an example project on GitHub as well.http://melomel.infoFull Disclosure: I'm the project owner of Melomel.

Ben Johnson 2010-10-15 16:48:18

What's your final decision on selecting automation tool? I'm now doing the same, not find a good one yet, thanks.

~Kathy

kathy chen 2010-10-11 21:11:40

We went with selenium with flex plugin. Its not perfect but its better than the alternatives. I am no longer personally involved with that but i know we had to add some cide to the flex plugin in order to support our custom widgets and co

Gregory Mostizky 2010-10-13 07:19:14

ansaurus

tags:

views:

answers:

Flex automated acceptance testing tools

related questions