views:

2100

answers:

10

I'm writing an application that will have multiple threads running, and want to throttle the CPU/memory usage of those threads.

There is a similar question for C++, but I want to try and avoid using C++ and JNI if possible. I realize this might not be possible using a higher level language, but I'm curious to see if anyone has any ideas.

EDIT: Added a bounty; I'd like some really good, well thought out ideas on this.

EDIT 2: The situation I need this for is executing other people's code on my server. Basically it is completely arbitrary code, with the only guarantee being that there will be a main method on the class file. Currently, multiple completely disparate classes, which are loaded in at runtime, are executing concurrently as separate threads.

I inherited this code (the original author is gone). The way it's written, it would be a pain to refactor to create separate processes for each class that gets executed. If that's the only good way to limit memory usage via the VM arguments, then so be it. But I'd like to know if there's a way to do it with threads. Even as a separate process, I'd like to be able to somehow limit its CPU usage, since as I mentioned earlier, several of these will be executing at once. I don't want an infinite loop to hog up all the resources.

EDIT 3: An easy way to approximate object size is with java's Instrumentation classes; specifically, the getObjectSize method. Note that there is some special setup needed to use this tool.

+4  A: 

You can get a lot of info about CPU and memory usage via JMX, but I don't think it allows any active manipulation.

For controlling CPU usage to some degree, you can use Thread.setPriority().

As for memory, there is no such thing as per-thread memory. The very concept of Java threads means shared memory. The only way to control memory usage is via the command line options like -Xmx, but there's no way to manipulate the settings at runtime.

Michael Borgwardt
+1  A: 

You can assign different priorities to the threads so the most relevant thread get scheduled more often.

Look at this answer to see if that helps.

When all the running thread have the same priority they may run like this:

t1, t2, t3,     t1, t2, t3,   t1, t2, t3

When you assign a different priority to one of them it may look like:

t1, t1, t1, t1,    t2,    t1, t1, t1 t3.

That is, the first thread runs "more often" that the rest.

OscarRyz
A: 

Thread.setPriority() may help, but it doesn't allow you to cap the CPU used by a thread. In fact, I've haven't heard of any Java library that does this.

It might be possible to implement such a facility provided that your threads are prepared to cooperate. The key is to have the threads periodically call into a custom scheduler, and have the scheduler monitor thread CPU usage using JMX. But the problem is that if some thread doesn't make the scheduler call often enough it may well exceed the throttling limits. And there's nothing you can do about a thread that gets stuck in a loop.

Another theoretical route to implementing would be to use Isolates. Unfortunately, you will be hard pressed to find a general purpose JVM that implements isolates. Besides, the standard APIs only allow you to control the isolate, not the threads within the isolate.

Stephen C
A: 

The only way you can limit Thread CPU usage is by either block on a resource or to call yield() frequently.

This does not limit CPU usage below 100% but gives other threads and processes more timeslices.

Thorbjørn Ravn Andersen
+10  A: 

If I understand your problem, one way would be to adaptively sleep the threads, similarly as video playback is done in Java. If you know you want 50% core utilization, the your algorithm should sleep approximately 0.5 seconds - potentially distributed within a second (e.g. 0.25 sec computation, 0.25 sec sleep, e.t.c.). Here is an example from my video player.

long starttime = 0; // variable declared
//...
// for the first time, remember the timestamp
if (frameCount == 0) {
    starttime = System.currentTimeMillis();
}
// the next timestamp we want to wake up
starttime += (1000.0 / fps);
// Wait until the desired next time arrives using nanosecond
// accuracy timer (wait(time) isn't accurate enough on most platforms) 
LockSupport.parkNanos((long)(Math.max(0, 
    starttime - System.currentTimeMillis()) * 1000000));

This code will sleep based on the frames/second value.

To throttle the memory usage, you could wrap your object creation into a factory method, and use some kind of semaphore with a limited permits as bytes to limit the total estimated object size (you need to estimate the size of various objects to ration the semaphore).

package concur;

import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;

public class MemoryLimited {
    private static Semaphore semaphore = new Semaphore(1024 * 1024, true);
    // acquire method to get a size length array
    public static byte[] createArray(int size) throws InterruptedException {
        // ask the semaphore for the amount of memory
        semaphore.acquire(size);
        // if we get here we got the requested memory reserved
        return new byte[size];
    }
    public static void releaseArray(byte[] array) {
        // we don't need the memory of array, release
        semaphore.release(array.length);
    }
    // allocation size, if N > 1M then there will be mutual exclusion
    static final int N = 600000;
    // the test program
    public static void main(String[] args) {
        // create 2 threaded executor for the demonstration
        ExecutorService exec = Executors.newFixedThreadPool(2);
        // what we want to run for allocation testion
        Runnable run = new Runnable() {
            @Override
            public void run() {
                Random rnd = new Random();
                // do it 10 times to be sure we get the desired effect
                for (int i = 0; i < 10; i++) {
                    try {
                        // sleep randomly to achieve thread interleaving
                        TimeUnit.MILLISECONDS.sleep(rnd.nextInt(100) * 10);
                        // ask for N bytes of memory
                        byte[] array = createArray(N);
                        // print current memory occupation log
                        System.out.printf("%s %d: %s (%d)%n",
                            Thread.currentThread().getName(),
                            System.currentTimeMillis(), array,
                            semaphore.availablePermits());
                        // wait some more for the next thread interleaving
                        TimeUnit.MILLISECONDS.sleep(rnd.nextInt(100) * 10);
                        // release memory, no longer needed
                        releaseArray(array);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
            }
        };
        // run first task
        exec.submit(run);
        // run second task
        exec.submit(run);
        // let the executor exit when it has finished processing the runnables
        exec.shutdown();
    }
}
kd304
I'd like to make this easier for people to look at quickly; would you mind adding a bit more detail as to what you're doing, and commenting the code?
Alex Beardsley
It won't help much as the sample code is already too long without comments. But I'll do it.
kd304
This is a really clever way to handle memory management, and is the only good direction anyone's given so far. As is it won't work for me because of EDIT 2 in my OP, but I'll probably end up using Aspects to intercept new objects getting created for that thread, and keep track of it with a semaphore for each Thread.
Alex Beardsley
The example code looks like it has moved - the new link is probably http://code.google.com/p/open-ig/source/browse/trunk/open-ig/src/hu/openig/VideoPlayer.java
Knubo
A: 

To reduce CPU, you want to sleep your threads inside of common if and while loops.

while(whatever) {
    //do something
    //Note the capitol 'T' here, this sleeps the current thread.
    Thread.sleep(someNumberOfMilliSeconds);
}

Sleeping for a few hundred milliseconds will greatly reduce CPU usage with little to no noticeable result on performance.

As for the memory, I'd run a profiler on the individual threads and do some performance tuning. If you out throttled the amount of memory available to thread I think an out of memory exception or starved thread is likely. I would trust the JVM to provide as much memory as the thread needed and work on reducing the memory usage by keeping only essential objects in scope at any given time.

James McMahon
+1  A: 

Care of Java Forums. Basically timing your execution and then waiting when your taking too much time. As is mentioned in the original thread, running this in a separate thread and interrupting the work thread will give more accurate results, as will averaging values over time.

import java.lang.management.*;

ThreadMXBean TMB = ManagementFactory.getThreadMXBean();
long time = new Date().getTime() * 1000000;
long cput = 0;
double cpuperc = -1;

while(true){

if( TMB.isThreadCpuTimeSupported() ){
    if(new Date().getTime() * 1000000 - time > 1000000000){ //Reset once per second
     time = new Date().getTime() * 1000000;
     cput = TMB.getCurrentThreadCpuTime();
    }

    if(!TMB.isThreadCpuTimeEnabled()){
     TMB.setThreadCpuTimeEnabled(true);
    }

    if(new Date().getTime() * 1000000 - time != 0)
     cpuperc = (TMB.getCurrentThreadCpuTime() - cput) / (new Date().getTime() *  1000000.0 - time) * 100.0;      
    }
//If cpu usage is greater then 50%
if(cpuperc > 50.0){
     //sleep for a little bit.
     continue;
}
//Do cpu intensive stuff
}
Andrew
A: 

If you run the threads in a separate process you can cap the memory usage and limit the number of CPUs or change the priority of these threads.

However, anything you do is likely to add overhead and complexity which is often counter-productive.

Unless you can explain why you would want to do this (e.g. you have a badly written library you don't trust and can't get support for) I would suggest you don't need to.

The reason its not easy to restrict memory usage is there is only one heap which is shared. So an object which is used in one thread is usable in another and is not assigned to one thread or another.

Limiting CPU usage means stopping all the threads so they don't do anything, however a better approach is to make sure the thread don't waste CPU and are only active doing work which needs to be done, in which case you wouldn't want to stop them doing it.

Peter Lawrey
Added the reasoning for this in the OP.
Alex Beardsley
An infinite loop will consume only one core. Many new servers have 4 to 16 cores so this might not be such a problem as it used to be. Note: an independent process can be safely killed for any reason.
Peter Lawrey
A: 

Why not instead of doing "threading" do cooperative multitasking, would be interesting to see if you can manipulate http://www.janino.net/ to run a program for a certain amount of time/set of intstructions, then stop and run the next program. At least that way its fair, give everyone the same time slice...

JH
A: 

If you run your foreign-code threads at a low priority, and run a profiler at high/normal priority, you may be able to observe when a particular Thread is not playing well in the sandbox. To profile the threads look at: www.javaworld.com/javaworld/javatips/jw-javatip92.html

[updated: this is now standard in JVM (Java 1.5), per above using java.lang.management: http://nadeausoftware.com/articles/2008/03/java%5Ftip%5Fhow%5Fget%5Fcpu%5Fand%5Fuser%5Ftime%5Fbenchmarking ]

Presumably you can then kill the annoying thread(s).

Jack Parker
The trouble with this is I don't want to kill an "annoying" thread if it's doing something legitimate. I guess if it's running for a full 30 seconds or something, I can reasonably assume it's not doing what it's supposed to, or the author should implement it more efficiently and I can therefore kill it. But I'd rather throttle the thread while its running for those 30 seconds, in case it is actually doing legitimate work, but just taking up a lot of resources.
Alex Beardsley