Are open source automated testing tools and frameworks better than commercial products ?

Question

About three years ago I switched from using commercial testing tools to using an open source testing framework (WatiN), and a UI automation framework I developed myself.

Since doing this I think life is much better using these libraries and Visual Studio, than expensive dedicated testing tools with either their own languages, or VBA.

Are open source automated testing tools and frameworks better than commercial products

Or do expensive commercial automated testing tools such as WinRunner, QuickTest Pro, Testpartner etc ... still have a future?

Answer 1

It's hard to make that sort of determination for a generic category like "testing tools". Usually, it's best to evaluate both the commercial solution and the open source solution on a case-by-case basis. From the wording of your question, it sounds like you've found the open source solution a better fit for your needs.

However, there are some points you can use in your decision making process:

Commercial Tool Benefits:

• Support - usually, the company is paying people to support the product. In addition, many commercial companies offer support contracts for various levels of support. If you need support in a crunch, commercial support is the way to go.

Open Source Tool Benefits

• Price - pretty obvious...it's hard to compete against free
• Openness - open source projects tend to adapt open standards more readily than commercial products (a lot of commercial products also adapt open standards, but open source software tends to do so more frequently).
• Self-support - If your company allows it, you can fix the bugs you find in the tools yourself. No need to wait for a third party to get around to fixing them.

It's also worth mentioning that a lot of the commercial testing tools are built on their open source counterparts. If that is the case, then you might be better off going with the open source versions, unless you need the added support.

I think you'll find that commercial products and Open Source products tend to have similar feature sets. In other cases, they may solve the problem with completely different approaches. Again, you'll probably want to make the evaluation for every case.

Answer 2

I got the experience, that tools for developers are the first and best class of software adopted in open-source. So you can see a big amount of great testing-tools in OS. So I think in most common environments you have test-frameworks in OS that work very well. But I do not code in every environment, so maybe you want to say which language/technology you use, and we can mention some good tools (OS or commercial).

Answer 3

All commercial tools have some +s's and -s's. Not all of them can be considered as an exact fit in an automation environment which an AUT requires. I have worked with all the big names it all depends on the automation Engineer how well He/She understands the concept and leverages the tool to the fullest. Last 5 years I came to the conclusion it is high time we divorce these Commercial tools and go after Open Source Why? I can customize the Open source to my needs. I know where and what went wrong and at the same time know how to fix it. I can customize things to the extent required write from planning to the execution to Reporting is in our control as automation engineers so creativity plays a role to implement the needs of the organization and the individuals who are affected by the automation testing.