Can you recommend low cost automated testing tools for a .NET Winforms application?

Question

Looking for an automated testing tool to support functional/regression testing for a .NET Winforms client server commercial application. The top tier products: HP QuickTest Pro, Borland SilkTest, IBM Rational Functional Tester and Compuware TestPartner are in the 5-10K price range. Would you recommend a lower cost tool set for functional/regression testing? Need tool that allows BA/QA analyst can be productively engaged test case development. So I am ruling out unit testing tools like xUnit which are entirely for programmer use. Yet I also want a tool that provides flexible scripting support to allow testing of a wider range of issues (possibly with developer support).

Please limit your answer to a single recommened tool. If your recommendation is already answered please upvote to create a ranking for tools available.

Thanks

Releated question: Tools for Automated GUI Testing on Windows

Answer 1

AutoIT/AutoHotKey is free (I use the two synonimously). They're scriping languages, your testers will need to know how to code the language. AutoIT uses VB so it's easy to pickup.

The way we use it is to replace users inputs by automating mouse mouse and clicks, and keyboard presses. You can watch (and wait) for windows to open, and them close them as desired. Essentially it just simulates what a human tester would do during intergration testing or full regression testing.

It might be a bit much to ask the BA/QA analyst to learn the scripting language, but then again it's not very complex.

Answer 2

TestComplete is a good choice. It has all the features at an affordable price. We are using it at our company here for a year and so far so good.

Answer 3

Fit/Fitnesse.

We have started to use Fit & Fitnesse for .NET. As an open source offering, price is right. Wiki based test development works well enough for our QA engineers. Writing fixtures requires a good amount developer support.
So getting the developers on board is a current struggle, as they are use to xUnit.

Answer 4

Maybe the software from Ranorex could help you? I've only played with it for a few hours while trying to evaluate it though: no long term experience with it.

Seems to me the product combination (recorder which generates VB/C#/python code against their automation API which - when it is run - replays the recorded actions) would work quite well for regression testing. Due to the nature with generating code and having to paste/integrate that into your own test runner framework however, you will definitely need a developer to support test development.

Answer 5

Use the Microsoft UI Automation portion of the .NET framework (System.Windows.Automation). It will automatically deal with things like the name of the class changing, the location/size of a button changing, etc. (In other words, all the things that traditional tools like SilkTest break on.) Plus you'll also be building in accessibility support (important if you want to sell to the U.S. government which requires being compatible with screen reader technologies, etc).

Answer 6

Would not traditional QTP work for all windows application (.NET) ? When the complexity of the code is very high and the consequent cost of manual testing is enormous, instead of manual testing automated tools should be used. QTP also works for Web applications and the problem occurs when it does not function properly for certain ASP.NET applications meaning you have to go into the code and figure out why QTP is not working properly. I only wonder how something like that is to be handled. It is called crisis management in which failure can lead to a catastrophic disaster.

Answer 7

One low cost tool is to write your own. That's what we have done.

We use IronPython to write our tests. It's a great language for that. It's a full .NET language, so it can call your .NET product code seamlessly, regardless of whether it's C# or whatever. We build our tests on top of the Python standard library module 'unittest'.

This works out-of-the-box for unit tests. For functional tests, we had to create a nontrivial framework, to allow our tests to fire up the application, and give our tests helper methods to click GUI buttons, type text in textboxes, etc.

Overall we're happy with the result.

Answer 8

Silk, Compuware, Mercury are not a good winforms automation development tools for Winforms. They all are good Web GUI testing tools. The very best is Visual Studio 2010. Use it and you will say Ahhhhhh... Wowwwwww.

Answer 9

Test complete is short of being good. We are using Eclipse TPTP for automation development for all windows applications. It is free, open source and solid.

Answer 10

For winforms automation/testing you should definately check out White!