How to get a command line process to use less processing power

Question

I am wondering how to get a process run at the command line to use less processing power. The problem I'm having is the the process is basically taking over the CPU and taking MySQL and the rest of the server with it. Everything is becoming very slow.

I have used nice before but haven't had much luck with it. If it is the answer, how would you use it?

I have also thought of putting in sleep commands, but it'll still be using up memory so it's not the best option.

Is there another solution?

It doesn't matter to me how long it runs for, within reason.

If it makes a difference, the script is a PHP script, but I'm running it at the command line as it already takes 30+ minutes to run.

Edit: the process is a migration script, so I really don't want to spend too much time optimizing it as it only needs to be run for testing purposes and once to go live. Just for testing, it keeps bring the server to pretty much a halt...and it's a shared server.

Answer 1

Using CPU cycles alone shouldn't take over the rest of the system. You can show this by doing:

while true; do done

This is an infinite loop and will use as much of the CPU cycles it can get (stop it with ^C). You can use top to verify that it is doing its job. I am quite sure that this won't significantly affect the overall performance of your system to the point where MySQL dies.

However, if your PHP script is allocating a lot of memory, that certainly can make a difference. Linux has a tendency to go around killing processes when the system starts to run out of memory.

I would narrow down the problem and be sure of the cause, before looking for a solution.

Answer 2

The best you can really do without modifying the program is to change the nice value to the maximum value using nice or renice. Your best bet is probably to profile the program to find out where it is spending most of its time/using most of its memory and try to find a more efficient algorithm for what you are trying to do. For example, if your are operating on a large result set from MySQL you may want to process records one at a time instead of loading the entire result set into memory or perhaps you can optimize your queries or the processing being performed on the results.

Answer 3

You should use nice with 19 "niceness" this makes the process very unlikely to run if there are other processes waiting for the cpu.

 nice -n 19 <command>

Be sure that the program does not have busy waits and also check the I/O wait time.

Answer 4

Which process is actually taking up the CPU? PHP or MySQL? If it's MySQL, 'nice' won't help at all (since the server is not 'nice'd up).

If it's MySQL in general you have to look at your queries and MySQL tuning as to why those queries are slamming the server.

Slamming your MySQL server process can show as "the whole system being slow" if your primary view of the system through MySQL.

Answer 5

You could mount your server's interesting directory/filesystem/whatever on another machine via NFS and run the script there (I know, this means avoiding the problem and is not really practical :| ).

Answer 6

You should also consider whether the cmd line process is IO intensive. That can be adjusted on some linux distros using the 'ionice' command, though it's usage is not nearly as simplistic as the cpu 'nice' command.

Basic usage:

ionice -n7 cmd

will run 'cmd' using 'best effort' scheduler at the lowest priority. See the man page for more usage details.