LR: Can I make the pseudorandom in LoadRunner deterministic?

Question

There are a couple of sources for random in LoadRunner scenarios:

• rand () function
• Random think time deltas (runtime settings)
• Random pacing time components (runtime settings)
• Random parameters (as part of the VUGen test)

I use those functionalities, and I could live with their pseudorandomness. I cannot, however, live with the fact that all scenario runs containing at least one of those functionalities behave pseudorandomly AND indeterministic, i.e. for a given start state (random seed), I want two runs to generate EXACTLY the same load, including timing (pacing and think times). So I want two runs to be based on EXACTLY the same random sequences. That means that I want to seed all random generators myself, as part of the initialization of each run.

I can use srand () to set a seed value for rand (). Setting a specific (hard-coded) seed value upon init always results in the same sequence delivered by rand () -- for all virtual users. If I seed with the VUser ID number, I would even get different rand () sequences for every vuser, while they still are the same from run to run for each user.

What about the other pseudorandom sources in LR, those beyond rand ()? Do I have a chance to seed them all so I get deterministic scenario behavior?

I think that would greatly help.

Without a mechanism like that, one has to plan for very long, and/or very high-traffic test scenarios in order to "average out" the randomness in the results statistics (do you agree with this?) which I do all day.

Answer 1

The short answer to your question is: NO.

Random implies just what it says => "Random". 

If you use the "built in" random features of parameters you are pretty much screwed as you have no control over how the internal random-seeds are initialized, and this can not predict in any way the next value.

If what you ultimately want to achieve is to extrapolate the results and predict server behavior under load you are in for a very rocky road.

Extrapolating results

Your run with 100 vusers and achieve an avg. of 50-60 hits/sec with
response times under 3 sec.

Logically 1000 vusers (10x load) would give you 500-600 hits/sec ... 

But what about the response times? How do you extrapolate them? How do you know
when the web-server(s) chokes and achieves it's knee-point? 

Remember that the hits/sec is directly proportional to the response times... so predicting hits/sec (or pages/sec) becomes very difficult and inaccurate

Things you can not control

Even if you'd achieve an "exact" copy of another run you still have to deal with the response times and network delays, that in effect are always different, regardless of circumstances (and also totally out of your control).

A more "realistic" way to define load

Load-testing in itself is not an exact science, and no load-test can ever simulate the real world completely, but we can get close. The way we do it here is that we try to simulate the individual users as close as possible. This way we can set the load expectations according to user-types, something that the "business" people usually have a clue about.

We also divide the "users" into types, such as power, normal or novice user - the difference for these are the speed at which they operate (and the way they use the UI's).

By doing this we can "load" the target application with a certain "expected user load" instead of pages/sec or hits/sec values or other technical meters.

We also execute longer runs to see how the service behaves over time, so a 72h or more test is not unusual for the Endurance test phase. This also shows if there are any memory leaks over time on the servers and how background processes impact the server performance during "night time"