VSTS Test Edition or HP's LoadRunner?

Question

I have had this debate with some peers off and on for a while. I am certified in the HP tools, but have been spending more and more time with VSTS Test Edition 2008. I am looking for opinions on what people think of the future of both products and how they compete.

LoadRunner's strengths include its vast array of protocols supported. Unfortunately since HP took over from Mercury, they are beginning to lag behind, especially in the new internet spaces.

VSTS Test, once very limited, is now quite impressive, especially in 2010. I don't know if it makes business sense, but I would love for VSTS Test to take on some additional protocols. Many of my clients would like to move away from HP and their licensing costs.

Finally, I am looking for good resources for VSTS Test. I have been playing with it, but would like to see some dedicated courses/material, instead of just a part of the larger VSTS.

Thanks!

Answer 1

"lag behind" is quite a generalization.

imho, VSTS, even at 2010, does not match LoadRunner's ease-of-use in system setup and power of results analysis.

However, VSTS Test Edition has a different pricing model than LoadRunner. Although Microsoft charges less for each VSTS end-user emulated, each script developer pays $4,000 (or so) for each VSTS desktop license. LoadRunner charges only by vuser on the server and does not require a license to create and test scripts at the desktop.

But the main cost with both products revolve around the amount of skill and time needed to create and debug scripts. Although LoadRunner (by default) uses C-language script functions from a vendor-specific library, LoadRunner does provide a mechanism that avoids the memory management and string handling hassles faced by native C programmers.

Both LoadRunner and VSTS face the challenge of providing mechanisms for being able to recognize the fast-changing landscape of front-end RIA libraries (JavaScript/AJAX, jQuery, Telerik, Infragistics, WPF, Silverlight, Adobe, etc.). New libraries and versions are coming out at a faster rate than HP can provide recognition / script generation capabilities.

Although HP LoadRunner claims it can emulate how browsers process JavaScript, many have had difficulty with making it work.

Consequently, performance script developers often need to have an understanding of how these various libraries work at a deeper level (more low-level) that most developers creating applications.

http://wilsonmar.com/ajax%5Frec.htm explains this in excruciating detail.

Answer 2

VS 2010 works beautifully with Silverlight, AJAX and a bunch of protocols used in RIAs. AJAX support is out of the box, however support for binary payload editing and parameterization is provided via a web plugin

Check out http://blogs.msdn.com/anutthara/pages/resources-for-test-tools-in-visual-studio-2010.aspx for more resources on VS 2010 Test Tools.

• Anu

Answer 3

"VS2010 works beautifully with Silverlight"

That is not accurate.

VS2010 does not support Silverlight yet. That is according to Microsoft. http://social.msdn.microsoft.com/Forums/en/vsautotest/thread/49a07111-7d41-435f-8ae1-114a281fa86b

Answer 4

I would compare LoadRunner to Microsoft VSTS (Visual Studio Test Suite) only in terms of Web (HTTP) protocol which is the only one VSTS supports. Using VSTS for any other protocol I would call "tool abuse".

I've used both tools on many project and I must say that VSTS WebTests are a pleasure to write. If the tests are recorded only as a WebTests (without generating code from them) maintenance becomes very easy.

I just fall in love with extraction rules and plug-ins. It provides very portable code fragments which could be reused on other projects. With a small extraction rules repository in my "toolbox" I can correlate most of the scripts without writing a single line of code which I personally think is an incredible.

Here is a link to a post about recording, correlating and parametrizing VSTS WebTests using Fiddler.

On the other hand LoadRunner wins by a long shot is results analysis. I think it will be few years before Microsoft catch up with HP's tool (if ever)

Thanks, Maciej