FP-intensive hyperthreading performance on latest Xeons

Question

We have recently purchased a dual Intel X5650 workstation to run a floating-point intensive simulation, under Ubuntu 10.04.

Each X5650 has 6 cores, so there are 12 cores in total. The code is trivially parallel, so I have been running it mostly with 12 threads, and observing approximately "1200%" processor utilization through "top".

HyperThreading is enabled in the BIOS, so the operating system nominally sees 24 cores available. If I increase the number of threads to 24, top reports approximately 2000% processor utilization - however, it does not appear that the actual code performance increases by 20/12.

My question is - how does HyperThreading actually work on the latest generation of Xeons? Would a floating-point intensive code benefit from scheduling more than one thread per core? Does the answer change if the working set is on the order of the cache size, as compared to several times larger, or if there are substantial I/O operations (e.g. writing simulation outputs to disk)?

Additionally - how should I interpret processor utilization percentages from "top" when hyperthreading is enabled?

Answer 1

With HT, the OS will schedule 2 threads to each core at the same time. The utilization reported by top is essentially just the average number of threads in the "running" state over its sampling interval (typically 1 second). Running threads are available for the CPU to execute, but may not be getting much work done, e.g. if they're mostly stalled on cache misses.

When a thread is blocked on real I/O -- network, disk, etc. -- the OS will deschedule it from the core and schedule some other ready thread, so HT won't help.

HT tries to get more utilization out of the math execution units without actually doubling very much hardware in the core. If one thread has enough instruction-level parallelism and doesn't miss cache much, then it'll mostly fill up the core's resources and HT won't help. For heavy FP apps with data that doesn't fit in cache, HT still probably won't help much, since both threads are using the same execution units (SSE math) and both need more than the full cache -- in fact it's likely to hurt since they'll be competing for cache and thrashing more. Of course it depends on exactly what you're doing and what your data access patterns look like.

HT mostly helps on branchy code with irregular and unpredictable access patterns. For FP-intensive code you can often do better with 1 thread per core and careful design of your access patterns (e.g. good data blocking).