How can I make sure N threads run at roughly the same speed?

Question

I'm toying with the idea of writing a physics simulation software in which each physical element would be simulated in its own thread.

There would be several advantages to this approach. It would be conceptually very close to how the real world works. It would be much easier to scale the system to multiple machines.

However, for this to work I need to make sure that all threads run at the same speed, with a rather liberal interpretation of 'same'. Say within 1% of each others.

That's why I don't necessarily need a Thread.join() like solution. I don't want some uber-controlling school mistress that ensures all threads regularly synchronize with each others. I just need to be able to ask the runtime (whichever it is---could be Java, Erlang, or whatever is most appropriate for this problem) to run the threads at a more or less equal speed.

Any suggestions would be extremely appreciated.

UPDATE 2009-03-16

I wanted to thank everyone who answered this question, in particular all those whose answer was essentially "DON'T DO THIS". I understand my problem much better now thanks to everybody's comments and I am less sure I should continue as I originally planned. Nevertheless I felt that Peter's answer was the best answer to the question itself, which is why I accepted it.

Answer 1

I'm not a threading expert, but isn't the whole point of threads that they are independent from each other - and non-deterministic?

Answer 2

You can't really do this without coordination. What if one element ended up needing cheaper calculations than another (in a potentially non-obvious way)?

You don't necessarily need an uber-controller - you could just keep some sort of step counter per thread, and have a global counter indicating the "slowest" thread. (When each thread has done some work, it would have to check whether it had fallen behind the others, and update the counter if so.) If a thread notices it's a long way ahead of the slowest thread, it could just wait briefly (potentially on a monitor).

Just do this every so often to avoid having too much overhead due to shared data contention and I think it could work reasonably well.

Answer 3

Two things has to happen in order to achieve this. You have to assure thah you have equal number of threads per CPU core, and you need some kind of synchronization.

That sync can be rather simple, like checking "cycle-done" variable for each thread while performing computation, but you can't avoid it.

Answer 4

Threads are meant to run completely independent of each other, which means synchronizing them in any way is always a pain. In your case, you need a central "clock" because there is no way to tell the VM that each thread should get the same amount of ... uh ... what should it get? The same amount of RAM? Probably doesn't matter. The same amount of CPU? Are all your objects so similar that each needs the same number of assembler instructions?

So my suggestion is to use a central clock which broadcasts clock ticks to every process. All threads within each process read the ticks (which should be absolute), calculate the difference to the last tick they saw and then update their internal model accordingly.

When a thread is done updating, it must put itself to sleep; waiting for the next tick. In Java, use wait() on the "tick received" lock and wake all threads with "notifyAll()".

Answer 5

I think you have a fundamental misconception in your question where you say:

It would be conceptually very close to how the real world works

The real world does not work in a thread-like way at all. Threads in most machines are not independent and not actually even simultaneous (the OS will use context-switching instead). They provide the most value when there is a lot of IO or waiting occurring.

Most importantly, the real-world does not "consume more resources" as more complex things happen. Think of the difference between two objects falling from a height, one falling smoothly and the other performing some kind of complex tumbling motion...

Answer 6

Please don't. Threads are an O/S abstraction permitting the appearance of parallel execution. With multiple and multicore CPU's, the O/S can (but need not) distribute threads among the different cores.

The closest thing to your scalability vision which I see as workable is to use worker threads, dimensioned to roughly match the number of cores you have, and distribute work among them. A rough draft: define a class ActionTick which does the updating for one particle, and let the worker thread pick ActionTicks to process from a shared queue. I see several challenges even with such a solution.

1. Threading overheads: you get context switching overhead among different worker threads. Threads by themselves are expensive (if not actually as ruinous as processes): test performance with different thread pool sizes. Adding more threads beyond the number of cores tends to reduce performance!
2. Synchronization costs: you get several spots of contention: access to the work queue for one, but worse, access to the simulated world. You need to delimit the effects of each ActionTick or implement a lot of locking/unlocking.
3. Difficulty of optimizating the physics. You want to delimit the number of objects/particles each ActionTick looks at (distance cut-off? 3D-tree-subdivision of the simulation space?). Depending on the simulation domain, you may be able to eliminate a lot of work by examining whether any changes is even needed in a subset of items. Doing these kinds of optimizations is easier before queueing work items, rather than as a distributed algorithm. But then that part of your simulation becomes a potential scalability bottleneck.
4. Complexity. Threading and concurrency introduces several cans of worms to a solution. Always consider other options first -- but if you need them, try threads before creating your own work item scheduling, locking and execution strategies...

Caveat: I haven't worked with any massive simulation software, just some hobbyist code.

Answer 7

I'd recommend not using threads wherever possible because they just add problems later if you're not careful. When doing physics simulations you could use hundreds of thousands of discrete objects for larger simulations. You can't possibly create this many threads on any OS that I know of, and even if you could it would perform like shit!

In your case you could create a number of threads, and put an event loop in each thread. A 'master' thread could sequence the execution and post a 'process' event to each worker thread to wake it up and make it do some work. In that way the threads will sleep until you tell them to work.

You should be able to get the master thread to tick at a rate that allows all your worker threads to complete before the next tick.

I don't think threads are the answer to your problem, with the exception of parallelising into a small number of worker threads (equal to the number of cores in the machine) which each linearly sequence a series of physical objects. You could still use the master/event-driven approach this way, but you would remove a lot of the overhead.

Answer 8

You'll need some kind of synchronization. CyclicBarrier class has what you need:

A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other. The barrier is called cyclic because it can be re-used after the waiting threads are released.

After each 'tick', you can let all your threads to wait for others, which were slower. When remaining threads reach the barrier, they all will continue.

Answer 9

I would make a kind of "clock generator" - and would register every new object/thread there. The clock will notify all registered objects when the delta-t has passed. However this does not mean you need a separate thread for every object. Ideally you will have as many threads as processors. From a design point of you could separate the execution of the object-tasks through an Executor or a thread-pool, e.g. when an object receives the tick event, it goes to a thread pool and schedules itself for execution.

Answer 10

Even with perfect software, hardware will prevent you doing this. Hardware threads typically don't have fair performance. Over a short period, you are lucky if threads run within +-10% performance.

The are, of course, outliers. Some chipsets will run some cores in powersaving mode and others not. I believe one of the Blue Gene research machines had software controlled scheduling of hardware threads instead of locks.

Answer 11

Working at control for motors i have used some math to maintain velocity at stable state. The system have PID control, proportional, integral and derivative. But this is analog/digital system. Maybe can use similarly to determine how mush time each thread must run, but the biggest tip I can give you is that all threads will each have a clock synchronization.

Answer 12

Erlang will by default try and spread its processes evenly over the available threads. It will also by default try to run threads on all available processors. So if you have enough runnable Erlang processes then you will get a relatively even balance.

Answer 13

I'm first to admit I'm not a threading expert, but this sounds like a very wrong way to approach simulation. As others have already commented having too many threads is computationally expensive. Furthermore, if you are planing to do what I think you are thinking of doing, your simulation may turn out to produce random results (may not matter if you are making a game).

I'd go with a few worker threads used to calculate discrete steps of the simulation.

Answer 14

As you mention, there are many "DON'T DO THIS" answers. Most seem to read threads as OS threads used by Java. Since you mentioned Erlang in your post, I'd like to post a more Erlang-centered answer.

Modeling this kind of simulation with processes (or actors, micro threads, green threads, as they are sometimes called) doesn't necessarily need any synchronization. In essence, we have a couple of (most likely thousands or hundreds of thousands) physics objects that need to be simulated. We want to simulate these objects as realistically as possible, but there is probably also some kind of real time aspect involved (doesn't have to be though, you don't mention this in your question).

A simple solution would be to spawn of an Erlang process for each object, sent ticks to all of them and collect the results of the simulation before proceeding with the next tick. This is in practice synchronizing everything. It is of course more of a deterministic solution and does not guarantee any real time properties. It is also non-trivial how the processes would talk to each other to get the data they need for the calculations. You probably need to group them in clever ways (collision groups etc), have hibernated processes (which Erlang has neat support for) for sleeping objects, etc to speed things up.

To get real time properties you probably need to restrain the calculations performed by the processes (trading accuracy for speed). This could perhaps be done by sending out ticks without waiting for answers, and letting the object processes reply back to each tick with their current position and other data you need (even though it might only be approximated at the time). As DJClayworth says, this could lead to errors accumulating in the simulation.

I guess in one sense, the question is really about if it is possible to use the strength of concurrency to gain some kind of advantage here. If you need synchronization, it is a quite strong sign that you do not need concurrency between each physics object. Because you essentially throw away a lot of computation time by waiting for other processes. You might use concurrency during calculation but that is another discussion, I think.

Note: none of these ideas take the actual physics calculations into account. This is not Erlang strong side and could perhaps be performed in a C library or whatever strikes your fancy, depending on the type of characteristics you want.

Note: I do not know of any case where this has been done (especially not by me), so I cannot guarantee that this is sound advice.