Hi,
When I run my multi-threaded code, the system (linux) sometimes moves the threads from one processor to another. As I have as many threads as I have processors, it invalidates caches for no good reasons and it confuses my tracing activities.
Do you know how to bind threads to processors, and why does a system would do this ?
Use sched_setaffinity (this is Linux-specific).
Why would a scheduler switch threads between different processors? Well, imagine that your thread last ran on processor 1 and is currently waiting to be scheduled for execution again. In the meantime, a different thread is currently running on processor 1, but processor 2 is free. In this situation, it's reasonable for the scheduler to switch your thread to processor 2. However, a sophisticated scheduler will try to avoid "bouncing" a thread between processors more than necessary.
You can do this from bash. There is a wonderful taskset command I acquainted in this question (you may also find valuable discussion on how scheduler should operate there). The command takes a pid of a process and binds it to the specific processor(s).
taskset -c 0 -p PID
binds the process with PID to processor (core) number 0.
What does it have to do with threads? To each thread is assigned an identifier with the same rights as pid, also known as "tid". You can get it with gettid syscall. Or you can watch it, for example, in top program by pressing H (some processes will split to many seemingly equal entries with different pids---those are threads).