CPU Affinity Masks (Putting Threads on different CPUs)

Question

I have 4 threads, and I am trying to set thread 1 to run on CPU 1, thread 2 on CPU 2, etc. However, when I run my code below, the affinity masks are returning the correct values, but when I do a sched_getcpu() on the threads, they all return that they are running on CPU 4.

Anybody know what my problem here is?

Thanks in advance!

#define _GNU_SOURCE
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <sched.h>
#include <errno.h>

void *pthread_Message(char *message)
{
    printf("%s is running on CPU %d\n", message, sched_getcpu());
}

int main()
{
    pthread_t thread1, thread2, thread3, thread4;
    pthread_t threadArray[4];
    cpu_set_t cpu1, cpu2, cpu3, cpu4;
    char *thread1Msg = "Thread 1";
    char *thread2Msg = "Thread 2";
    char *thread3Msg = "Thread 3";
    char *thread4Msg = "Thread 4";
    int thread1Create, thread2Create, thread3Create, thread4Create, i, temp;

    CPU_ZERO(&cpu1);
    CPU_SET(1, &cpu1);
    temp = pthread_setaffinity_np(thread1, sizeof(cpu_set_t), &cpu1);
    printf("Set returned by pthread_getaffinity_np() contained:\n");
    for (i = 0; i < CPU_SETSIZE; i++)
        if (CPU_ISSET(i, &cpu1))
            printf("CPU1: CPU %d\n", i);

    CPU_ZERO(&cpu2);
    CPU_SET(2, &cpu2);
    temp = pthread_setaffinity_np(thread2, sizeof(cpu_set_t), &cpu2);
    for (i = 0; i < CPU_SETSIZE; i++)
        if (CPU_ISSET(i, &cpu2))
            printf("CPU2: CPU %d\n", i);

    CPU_ZERO(&cpu3);
    CPU_SET(3, &cpu3);
    temp = pthread_setaffinity_np(thread3, sizeof(cpu_set_t), &cpu3);
    for (i = 0; i < CPU_SETSIZE; i++)
        if (CPU_ISSET(i, &cpu3))
            printf("CPU3: CPU %d\n", i);

    CPU_ZERO(&cpu4);
    CPU_SET(4, &cpu4);
    temp = pthread_setaffinity_np(thread4, sizeof(cpu_set_t), &cpu4);
    for (i = 0; i < CPU_SETSIZE; i++)
        if (CPU_ISSET(i, &cpu4))
            printf("CPU4: CPU %d\n", i);

    thread1Create = pthread_create(&thread1, NULL, (void *)pthread_Message, thread1Msg);
    thread2Create = pthread_create(&thread2, NULL, (void *)pthread_Message, thread2Msg);
    thread3Create = pthread_create(&thread3, NULL, (void *)pthread_Message, thread3Msg);
    thread4Create = pthread_create(&thread4, NULL, (void *)pthread_Message, thread4Msg);

    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);
    pthread_join(thread3, NULL);
    pthread_join(thread4, NULL);

    return 0;
}

Answer 1

You're trying to set the affinity of threads that you did not initialize.

Edit: Ok, let me give you some more info:

Don't mix thread handles (the thing you store in the pthread_t variable) and what they represent (a thread of execution that runs somewhere). What you were trying to do is to set a property of a thread before it starts, with an API that requires the thread object. As it happens pthread_create creates the object and starts the execution at the same time, so trying to use pthread_setaffinity_np is not the right way to go (this is useful if you want to change the affinity of a currently running thread).

But... pthread_create has an attribute parameter (you're passing NULL to it). This is storing the information of how you want the thread to be created.

Affinity is one of the attributes you can set through that parameter. See the documentation here and here for how exactly.

Answer 2

I think the easiest would be to give the CPU mask as a parameter to each thread and have the thread request given affinity itself, as in example here: pthread_setaffinity_np(3).