Physical location aware user space memory allocation in Linux (memory controller affinity)

Question

I want to test the performance variations that may happen if memory is allocated and accessed from different physical CPUs and from different embedded memory controllers for a 64 bit, 2 CPU, 16 core Intel Xeon 5500 CPU based server. (Dell T710)

Looking at the vendor white paper i can see each physical CPU has 3 independent memory controllers. There are also 2 physical CPUs, for a total of 6 parallel memory controllers. I would like to know the performance of memory copies to/from the same controller, different controllers within the same physical CPU and between different physical CPUs.

To do this i need to set the CPU affinity of the process (can do) and memory affinity to the physical CPU (seems to be inherited from the cpu affinity) and then memory affinity to the memory controller (don't know!).

Does anyone know how to do this on Linux?

I assume that the physical address would correspond to the memory controller, so alternatively is there any way to request a physical memory range for a user space memory request in Linux?

Answer 1

The Linux kernel Documentation/numastat.txt mentions the numactl package for managing memory controllers: http://linuxmanpages.com/man8/numactl.8.php