Suggestions/Tricks for Throttling a PHP script

Question

I have a scheduled task that runs a script on a regular basis (every hour). This script does some heavy interaction with the database and filesystem and regularly takes several minutes to run. The problem is, the server's cpu-usage spikes while the script is running and slows down normal operations. Is there a way to throttle this process so that it takes longer but does not consume as many resources?

I've looked at different configuration options for PHP but there does not appear to be any that fit my needs.

Setting memory_limit in php.ini to something lower causes my data objects to overflow quite easily.

I've seen similar posts where people suggested using sleep() at certain points in the script but that does not prevent the script from spiking the server.

The optimal solution would be some way to tell the Lamp (in this case Wamp) stack to only use 10% max cpu utilization. I'm not concerned at all about runtime and would prefer that it take longer if it means saving cpu cycles per second. My alternate solution would be to setup a different server with database replication so the cron could go to town without slowing everything else down.

Environment: Windows Server 2k3, Apache 2.2.11, PHP 5.2.9, MySQL 5.1

I appreciate any insight to this situation.

EDIT: I appreciate all the answers, even the ones that are *nix-specific. It's still early enough in my situation to change the hosting environment. Hopefully this question will help others out regardless of the OS.

Answer 1

This might be a difficult change but it may worth it refactoring your data structures into iterators. Also, if you have circular references in your code, provide a method like clearReferences() that unsets these objects. This is a problem that is solved in PHP 5.3 by the way.

So if you have:

class Row
{
    protected $_table;

    public function __construct($table)
    {
        $this->_table = $table;
    }
}

class Table
{
    protected $_row;

    public function __construct()
    {
        $this->_row = new Row($this);
    }
}

Add a clearReferences() method to the Row class:

class Row
{
    public function clearReferences()
    {
        $this->_table = null;
    }
}

That's all I can come up with for the moment.

Answer 2

You can set processes in Windows to be a lower priority. I'm not sure how the process is being kicked off, but if you set the process to be a low priority, whatever wants CPU resources will get them if you set the priority to be really low.

Answer 3

If you have it(Apache) running as a service, you can change the priority settings in the Win control center /services. Your CPU usage will spike anyway, but other programs will be preferred by the scheduler. Also try putting the database/server on a different hd than your Applications.

Answer 4

This is a tricky problem. If you're running the PHP script via the command line, you can set the process's scheduling priority to low (start /low php.exe myscript.php I believe). If your PHP script itself is actually doing most of the processing that's eating your CPU, this might work. However, you said you are doing some heavy database and filesystem interaction, which this solution will not help. It looks like there is a MySQL hint "LOW_PRIORITY" for INSERT and UPDATE queries that may help you there, but I have not tried those.

Answer 5

Can you alter your cron entry to launch your script using nice?

Answer 6

Not a good idea to use a server for serving clients and analyse data.

So if you are looking for a final solution, make a few redesign of your application and offload the data analysis from the frontends and the live database to another system dedicated to this task.

Even if you can successfully throttle the analyzer, it would use up precious resources otherwise would be available to serve the users.

Answer 7

I have a bunch of scripts that I run from cron in a similar way using nice:

0 * * * * nice -n19 php myscript.php

This won't help the RAM utilization (only changing the way the script is written can do that), but it only uses the CPU that would otherwise be idle.

EDIT: didn't see that the question involved a Windows environment, sorry... leaving this in for any *nix users having the same problem..

Answer 8

In UNIX (LAMP) I managed to solve the problem by checking the load of the server before continuing the loop

function get_server_load($windows = 0) {
 $os = strtolower(PHP_OS);
 if(strpos($os, "win") === false) {
  if(file_exists("/proc/loadavg")) {
   $load = file_get_contents("/proc/loadavg");
   $load = explode(' ', $load);
   return $load;
  }
  elseif(function_exists("shell_exec")) {
   $load = explode(' ', `uptime`);
   return $load;
  }
  else {
   return "";
  }
    }
}

for(... ... ...){
 $data = get_server_load(); 
 if($data[0] < 0.2){
  // continue
 }else{
  sleep(1);
 }
}

This function should work also on windows but I can't guarantee it. On linux it gives you back an array with the load of the last 1 minute, 5 minutes and 15 minutes

Also, consider to start your scripts (if by CLI) with a lower priority (in Linux, use "nice")

You can also use other values before continuing the loop, like the number of Apache active processes (you can parse the page 127.0.0.1/server_status?auto if you enabled the mod_status in httpd.conf), or also the MySQL situation (active connections?)

Answer 9

Perhaps what your script is simply trying to do too much all at once. Would it do less if it ran three times an hour?

Another solution might be to setup an additional server just for running this sort of 'backend' processing. This would particularly effective if it is not putting undue load in the database, just the web server.

Yet another approach to look at is whether it's work can be divided in a different direction. These sorts of scripts often have a few big SQL statements that generate results used to generate a whole lot of little SQL statements. If the latter could be put aside somewhere, they can be run against the database as a later step. Such an approach might also let you use an unbuffered query to fetch the pre-processing data which could cut down significantly on memory consumption by the PHP code.