Is using processor usage for the automated scheduling of processes a good practice?

Question

I've been having a debate of sorts with a co-worker who suggested that we allow some cpu intensive processes in our enterprise to poll CPU usage and execute their tasks when the CPU usage is low. My counter-point was that while cpu usage in an ideal system would denote the level of system activity on a given server, in actuality it has too much inconsistency(peaks and dips over a short time) in a real system to be an effective indicator of when a cpu intensive process should run. And in addition I stated that the OS is designed to manage processor contention between threads and applications already. My suggestion was simply to run the process afterhours to avoid degrading the user's experience during the day.

My question is, can cpu usage be an effective indicator as to when processes should run in an enterprise setting? It would be a nice-to-know if I'm right, sort of right, or just being incorrect...

Edit: These applications are .NET services as well as SQL Server scheduled jobs.

Answer 1

No it isn't (so yes you are correct).

There are many ways that it can cause problems, off the top of my head:

• The OS is attempting to balance resource allocation. In order to do this it has a scheduling algorithm that uses a view of the current resource usage. What you describe is running a second scheduling algorithm that will fight with the first one (the OS scheduler) over allocation of CPU. This can cause strange feedback effects.

• Just using process usage doesn't take into account other resources such as memory. When one process runs it doesn't just display other processes by using processor cycles. It's working set of data is fighting with other processes to be kept in memory and out of swap. You can seriously degrade the performance (and latency especially) of other processes if you activate your task because the CPU is not in use and it causes their data to be paged out.

• Why reinvent the wheel? This is precisely what priority levels / idle-processing were invented for. If you just want your process to take up background CPU then set it running at the lowest priority level and allow the OS to schedule it.