How would you recommend doing stress testing to an azure cloud service?
A:
Why not just write some unit tests that issue repetitive calls to it? You could use the multi threading or even the new Parallel Programming library in .NET 4 to hammer it with many concurrent calls. Like Josh said, though, it's the cloud.
Steve Danner
2010-09-08 16:18:28
Because the bottleneck will be the network itself and not the tested application.
Adibe7
2010-09-08 18:29:15
If you are passing very little data across the wire in your initial request, outbound network limits should not matter. But if you are, then yes, that would become a problem. Can't tell that from the original post, however.
Steve Danner
2010-09-08 18:44:35
+1
A:
The issue you'll run into is that of unknown network issues (speed, latency, etc.). If you attempt to run a stress test from your own environment, you will need adequate outbound bandwidth and a known SLA. Also: You'll want to focus on a known product, rather than rolling your own; otherwise you won't know if you're hitting contention points with your load-generation, virtual-client management, reporting, client ramp-up issues, balancing multiple click-path scenarios, dealing with think-times, dealing with form-filling, etc. If you're serious about load and stress testing, go with a load-testing product.
I've had success testing Azure-hosted sites with LoadStorm - their solution is a cloud-based stress-testing tool (ironically on the Amazon EC2 platform). While their solution is not as powerful and customizable as, say, Microsoft's Load Test, I believe it provides all of the tasks required for a valid load/stress test (easily covers the 80/20 rule). I've been doing load- and stress-testing for years, with Microsoft's product as well as RadView's WebLOAD (similar to LoadRunner). While all of those tools provide a complete programmatic model, none are as straightforward to build and deploy as LoadStorm, and licensing is on a monthly basis (free for 25 concurrent virtual users if you want to try it out). Full pricing here.
Note: I have no affiliation with LoadStorm, aside from being interviewed by them regarding my load-testing experience.
David Makogon
2010-09-08 17:29:10
A:
That is a pretty broad question - since the Azure host could be providing a wide variety of services. I'll assume it is providing a website and/or web services (SOAP etc. over HTTP). Testing it is not significantly different than testing any other web site or service.
- Determine testing goals -- i.e. what question are you trying to answer?
- Choose one or more user scenarios that will help answer #1
- Determine performance requirements for scenarios in #2
- Start with the most commonly used scenario.
- Choose a testing tool suitable for simulating that scenario such as Web Performance Load Tester (disclaimer: I work for Web Performance)
- Run a test up to the expected load for that scenario and report the results back to mgmt/developers.
- Add more scenarios into your testing mix until you're done or run out of time/budget.
If you are new to load testing, this article may be helpful: Introduction to Load Testing
CMerrill
2010-09-09 18:48:02
A:
I'll second that vote for http://loadstorm.com/ load testing. No better way to test the cloud than to use a WHOLE BUNCH of machines to test it. There are other options out there too for cloud load testing.
Also check out these links. http://www.cloud-intelligence.com/cloud/testing http://www.webperformanceinc.com/load_testing/blog/tag/ec2/
Both have some great material on load testing. If you have questions, these two sources will get you pointed in the direction you need to be going.
Adron
2010-09-14 18:23:51