different behavior when using different number of multicoring workers

Question

I am playing around a bit with my program (trying to multicore a few parts) and I've noticed the "CPU history" looks a bit different, depend on how many workers I start. 2-4 workers seems to produce a "stable" workflow, however pegging 5-8 workers produces erratic behavior (from zero to max, see pictures). I should point out that all runs started out with "smooth" max capacity (e.g. 2 cores with only 25%), and started to exhibit erratic behavior only after a minute or so. What's going on? I have 4 core processor, and do you think this behavior may be related to this fact?

I hope you can see the pics.

2 workers

3 workers

4 workers

5 workers

6 workers

7 workers

8 workers

Answer 1

This is classic behaviour on a quad-core with hyperthreading. R doesn't gain with hyperthreading, as it uses often the complete core for the calculations. Thus, if one physical processor makes 2 logical ones, the processor has to switch continuously between both threads, which explains the patterns.

For R, I put off the hyperthreading on my computer. It just doesn't help, in contrary. When working with only one thread as R does normally, you lose capacity (max is 12.5% instead of 25%).