pthreads : allowed number of threads

Question

Hi,

I have been working now for few days on a small C program which uses pthreads. I spent more or less all yesterday looking for a deadlock bug, but now I have found out that the problem is not really a deadlock bug. The following piece of code has exactly the same problem.

#include <stdlib.h>
#include <pthread.h>
#include <semaphore.h>
#include <stdio.h>
#include <unistd.h>
#define NTHREADS 507

pthread_mutex_t runningThreadsMutex;
pthread_cond_t runningThreadsCond;
int runningThreads = 0;

void* HelloWorld(void* arg) {
  sleep(1);

  pthread_mutex_lock(&runningThreadsMutex);
  runningThreads--;
  printf("End thread %d\n", runningThreads);
  pthread_cond_signal(&runningThreadsCond);
  pthread_mutex_unlock(&runningThreadsMutex);

  return NULL;
}

int main() {
  pthread_t thread;

  pthread_mutex_init(&runningThreadsMutex, NULL);
  pthread_cond_init(&runningThreadsCond, NULL);

  for (int i = 0; i < NTHREADS; ++i) {
    pthread_mutex_lock(&runningThreadsMutex);
    printf("Create thread %d\n", runningThreads++);
    pthread_mutex_unlock(&runningThreadsMutex);
    pthread_create(&thread, NULL, HelloWorld, NULL);
  //  pthread_detach(thread);
  }

  pthread_mutex_lock(&runningThreadsMutex);
  while(runningThreads > 0) {
    pthread_cond_wait(&runningThreadsCond, &runningThreadsMutex);
  }
  pthread_mutex_unlock(&runningThreadsMutex);
  return 0;
}

The code above seems to work nicely on my laptop (64 bit linux machine) for NTHREADS < 506. In that case it prints out something like this:

Create thread 0
Create thread 1
.
.
.
Create thread 505
End thread 505
End thread 504
.
.
.
End thread 0

---

And terminates as it should. However if I use NTHREADS >= 506, e.g. NTHREADS = 510 I get

Create thread 0
Create thread 1
.
.
.
Create thread 509
End thread 509
End thread 508
.
.
.
End thread 4

---

where it halts without ever terminating. So it seems like the last four (510-506=4) threads never terminate (or never start at all?).

I tried this code as well on an old 32 bit linux machine. There I get the same behaviour, except that it works well for NTHREADS < 382 but not for NTHREADS >= 382 (instead of 506).

When I googled for a solution I also found this question: http://bytes.com/topic/c/answers/728087-pthreads-limit, where someone is having the same problem when using pthread_join (which might be more natural when working with pthreads) but they don't give any good explanation.

Can anyone explain to me what I am doing wrong and what is the fundamental problem with this code? I guess this must be some kind of limit on the number of allowed threads, but how should I deal with it?

Answer 1

You need to check the return value for pthread_create. If it is non-zero then the function failed to create the thread. A typical problem will be running out of memory for the stacks for the new threads. e.g. with a 1Mb stack per thread the system will need at least 510Mb of free memory to start 510 threads.

Why are you running so many threads? Unless you have a massively parallel system with hundreds of processors then these threads will just compete for CPU time and other resources. You are probably better off with fewer threads (of the same order of magnitude as the number of processors in your system) that do the work in the most appropriate order.

Answer 2

Adding to Anthony's answer, you can reset the stack allocation for your threads using the following piece of code:

pthread_attr_t threadAttr;
size_t threadStackSize = 65536;   // this is the stack size in bytes, 
                                  // must be over 16384 for Linux 
pthread_attr_init(threadAttr);
pthread_attr_setstacksize(&threadAttr,threadStackSize);

    if( pthread_create(&threadId,&threadAttr,funcn,NULL) != 0 )
    {
        printf("Couldn't create thread\n");
        exit(1);
    }