Experiences with Test Automation FX

Question

Looking to add UI testing to my WinForms 3.5 project. Currently using MSTest for unit testing and MSBuild to build it.

One option I am looking at is Test Automation FX.

The product seems to be a bit new and not fully polished, but it seems to work. So, I'm curious if anyone else is using and has good or bad things to say about it.

It is quite a bit cheaper in price ($450) than Test Complete ($2000), so I also am trying to figure out what is lacking or missing, if anything, from Test Automation FX.

Answer 1

The company I work for uses SilkTest, which works very good. In general, when using automated testing, you would be doing lots of regression testing. What is more important is when you've modified an existing project, then the test software must still be able to run those tests without any errors. (Or, with the errors you'd expect.)

But the market does have lots and lots of other test solutions. In the past, I even saw a test setup which required two computers and additional hardware. The hardware would connect to the monitor, mouse and keyboard of the test system. The other end would connect to a special extension card in the test server. The hardware was there so the server could send keyboard commands to the test system and record anything that happened on the screen. With some additional OCR software, it was very well capable of analysing any errors. Then again, it had a price of six digits and to be honest, I'd rather buy a Porsche for that price and probably would have some cash left to bring two beautiful dates with me while driving through the boulevards in Nice, France...

There's a Wiki page with an overview of all kinds of test software. It doesn't compare them but you can find Test Automation FX there, although it doesn't provide much information. It seems limited to testing Windows GUI's only. TestComplete provides more information. Then again, comparing the Wiki's it also supports a lot more. Really a lot more. Enough to explain why it's that expensive...

Answer 2

Ranorex might be interesting for you. It is a Windows GUI test automation framework for testing many different application types including Winforms, WPF, Silverlight (and much more...). You can capture/replay your UI tests, edit your actions and generate C#, VB.NET and IronPython code.

http://www.ranorex.com

Answer 3

I have just starting to evaluate different GUI automate testing tool. I have looked at Test Automation FX, Ranorex and TestComplete. And the price for the software are in that order.

This is some of my conclusions:

Test Automation FX - Coded in C#, Fully VS integrated. But very slow in finding components and takes much memory and don't fully support DevExpress components

Ranorex - Coded in C#, Have a studio for maintating test but can be fully integrated into VS. Has better object support. And you can find objects in your software by regex expresseion on several thing. Have some problem with DevExpress components but is rather fast to work with.

TestComplete - Uses its on script language. VBscript is the easiest one (C#Script is just awkward notation). This have really good support for DevExpress components and runs the test really fast. But is very expemsive

Right now I don't know which I should use. Ranorex is alite better than Test Automation FX but both lack the full support for DevExpress components. TestComplete is nice but it introduce a new language to the development and is very expensive. But the test scripts are small and the program have more logic in finding very to click.

Answer 4

I have evaluated Test Automation Fx, Although it recognizes all the controls of my application (we use 3rd party controls from infragistics ie netAdvantage controls for WPF) It is very slow in recognizing the controls and even playback time is quite slow compared to QTP or Ranorex. I would recommend Ranorex over Test Automation Fx.