Correlation between requests per second and response time?

Question

Can someone please explain the correlation between requests per second and response time? Which are you trying to improve at first? If your competitor offers less 'requests per second' on his most used functionality then you, is your application performing better in terms of end-user performance?

Answer 1

It all depends on what sort of load your system will be under.

If you have millions of users then you need to handle more requests per second possibly at the expense of response time otherwise users may not be able to connect when they want to.

However, if you are only going to have 30 users then it's more important to them that your system responds quickly than it being able to handle a thousand requests a second.

Answer 2

If your competitor offers less 'requests per second' on his most used functionality then you, is your application performing better in terms of end-user performance?

I wouldn't agree with that. Look at Google. They make thousands of requests a second - hell, I think it's something like 100 million per day and 3 billion per month.

To answer your question, I think response time is more important than requests per second. Sure you can optimize/minimize the number of requests made, but if your product scales to handle unlimited requests (just by throwing more hardware at the problem) then I think that is more valuable.

Answer 3

Requests per second may be high while offering an awful user experience. You might have a lot of users buying thousands of concert tickets per second but the response time for each user is over 30 seconds.

For a high performing, enjoyable web site, you need to have a high number of requests per second and a maximum response time. As a user, I like 5 seconds or less.

Answer 4

Can someone please explain the correlation between requests per second and response time?

Think of this situation as if it were a gas station. Cars arrive at various intervals and occupy a pump; they spend some time filling up, and then they leave.

• Each car that arrives and occupies a pump is a request.
• The time it takes to fill up is your response time.

You can improve things in two ways:

• If you add more pumps, you can service additional cars at once because there will be more capacity.
• If you make all your pumps faster, you can service more cars over time with the same number of pumps, because each car will finish sooner.

Which are you trying to improve at first?

That depends. Do you want to serve people faster (improving their experience while making some others wait) and thus more people overall, or do you want to serve more people at once (at the possible expense of request time)? Ideally, get both metrics as good as possible.