views:

59

answers:

2

The following code is supposed to make 100,000 threads:

/* compile with:   gcc -lpthread -o thread-limit thread-limit.c */
/* originally from: http://www.volano.com/linuxnotes.html */

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>

#define MAX_THREADS 100000
int i;

void run(void) {
  sleep(60 * 60);
}

int main(int argc, char *argv[]) {
  int rc = 0;
  pthread_t thread[MAX_THREADS];
  printf("Creating threads ...\n");
  for (i = 0; i < MAX_THREADS && rc == 0; i++) {
    rc = pthread_create(&(thread[i]), NULL, (void *) &run, NULL);
    if (rc == 0) {
      pthread_detach(thread[i]);
      if ((i + 1) % 100 == 0)
    printf("%i threads so far ...\n", i + 1);
    }
    else
    {
      printf("Failed with return code %i creating thread %i (%s).\n",
         rc, i + 1, strerror(rc));

      // can we allocate memory?
      char *block = NULL;
      block = malloc(65545);
      if(block == NULL)
        printf("Malloc failed too :( \n");
      else
        printf("Malloc worked, hmmm\n");
    }
  }
sleep(60*60); // ctrl+c to exit; makes it easier to see mem use
  exit(0);
}

This is running on a 64bit machine with 32GB of RAM; Debian 5.0 installed, all stock.

  • ulimit -s 512 to keep the stack size down
  • /proc/sys/kernel/pid_max set to 1,000,000 (by default, it caps out at 32k pids).
  • ulimit -u 1000000 to increase max processes (don't think this matters at all)
  • /proc/sys/kernel/threads-max set to 1,000,000 (by default, it wasn't set at all)

Running this spits out the following:

65500 threads so far ...
Failed with return code 12 creating thread 65529 (Cannot allocate memory).
Malloc worked, hmmm

I'm certainly not running out of ram; I can even launch several more of these programs all running at the same time and they all start their 65k threads.

(Please refrain from suggesting I not try to launch 100,000+ threads. This is simple testing of something which should work. My current epoll-based server has roughly 200k+ connections at all times and various papers would suggest that threads just might be a better option. - Thanks :) )

A: 

One possible issue is the local variable thread in the main program. I think that pthread_t would be 8 bytes on your 64-bit machine (assuming 64-bit build). That would be 800,000 bytes on the stack. Your stack limit of 512K would be a problem I think. 512K / 8 = 65536, which is suspiciously near the number of threads you are creating. You might try dynamically allocating that array instead of putting it on the stack.

Mark Wilkins
+1  A: 

pilcrow's mention of /proc/sys/vm/max_map_count is right on track; raising the value allows more threads; not sure of the exact formula involved, but a 1mil+ value allows for some 300k+ threads.

(For anyone else experimenting with 100k+ threads, do look at pthread_create's mmap issues... making new threads gets really slow really fast when lower memory is used up.)

rekamso