TFS Load Testing Web Tests

Question

I am configuring a load test and am curious/confused on settings. I am testing an intranet website, that is expected to have 6000 concurrent users. My employer had some previous consultant tell them that the load test users does not matter and that we need to worry about requests/second. They have previously determined that those 6000 users would generate 30 rps, while I feel that is not correct we need to show that we can exceed that number. The previous load test was set for only 200 users and the results showed that it did exceed the 200 rps. They were happy with the results, but that is not how I understand this.

My question is, if we need to support 6000 concurrent users should I just set my users to 6000 and run, or is the rps an adequate piece of data to rely on?

Any help is appreciated. Thanks

Answer 1

It is really hard to measure the apples of a "Virtual User" with the orange that is a real person. A real person may take seconds to minutes to read a webpage and then take some action. A virtual user will be able to process a webpage every few seconds.

To test adequately you need to figure out a common unit of "work" between real users and the load we can generate with Visual Studio. The consultant probably recommended that RPS be used as it is easy to measure from any loadtest with whatever webtests inside it. It is a good measure.

The accuracy of the RPS measure rests on the assumptions made about your users. The math works a little like: I have 6000 users, who need to use the site every day. Mostly they log in in the mornining, work a bit before morning tea and hit the site more heavily from 2pm-3:30pm. Say

Looking at previous logs for a site or just guessing you can say: Maybe at peak a users hits the site every minute or so.

Figuring at peak site usage 30% of the users are working. So Users:6000 Peak percentage: 30% RPS/users: 1/60

6000 * 30% * 1/60 = 30 RPS.

So if the site can process 200RPS we can roughly say it is equivalent to all 6000 users hitting the site for a page every minute.

6000 * 100% * 1/60 = 200 RPS.

When you change the assumptions about your real users, the number of RPS changes, often dramatically.