maximize load but don't bring it to a crawl

Question

I have a shell script that runs very cpu intensive programs. FFMPEG,ffmpeg2theora,etc. and I want to be able to run them but not choke the server. Is there something I can do to make sure the running programs are running as fast as possible but not hurting the server?

Like a priority system...if something else comes along that needs it the other programs drop in priority aka cpu usage. I know there is "nice" but with the above programs are not working with it.

I played with cpulimit but that makes me say it can't go higher than that even though there may be a light load and it could handle more.

Any existing unix tools or ideas?

Answer 1

I know there is "nice" but with the above programs are not working with it.

Can you define "not working"? Setting a nice value of 20 should allow the program to use as much CPU time as possible, yielding to anything of a higher priority that needs the CPU.

What nice doesn't do is throttle I/O. Video conversion is kind of I/O dependent. For that, there's ionice, but that comes with some caveats.