The relationship between cores and the number of threads I can spawn

Question

I have a Intel Quad Core CPU.

If I was to develop a Winforms application which only be used on my machine (I use C# btw), how many threads can I spawn?

Is there some sort of correlation between cores and the max number of threads I can have running at any one time? Would I need to find out how many threads are running at any one time and if so, is this possible? (I know there's properties like min and max threads)? Would this depend on the threadpool (does the max no of threads in this pool change?). This is the C# part of this post/thread.

Answer 1

It all depends, if your threads are active (and not waiting for IO) 100% of the time then there is little point in having more that 1 thread per CPU. However, this is rarely the case unless you are performing complex numeric calculations.

.Nets threadpool has: http://msdn.microsoft.com/en-us/library/system.threading.threadpool.aspx

The thread pool has a default size of 250 worker threads per available processor, and 1000 I/O completion threads.

So, I would say, there is very little recommendations anyone can give you, besides:

• Measure measure measure.

At some point when you add more threads stuff will get slower, due to context switching and synchronization.

Answer 2

While there is a loose correlation between threads and cores (the only way for threads to execute truly concurrently would be for them to run on separate cores, but that knowledge is of less value than you might think), the real work is done by the operating system scheduler, in this case the thread scheduler in Windows.

As to how many threads you can create, that will vary from system to system. The ThreadPool class does not place any restrictions on spawning your own threads; it has a, well, pool of threads that it manages itself internally. Those are the values you can see when inspecting the properties of the ThreadPool class. That is not to say, however, that you should spawn limitless threads; eventually the OS will be spending more time switching between your threads than it will spend actually allowing your threads to run. Figure out how many threads are appropriate for your application through benchmarking.

What exactly are you trying to do?

Answer 3

how many threads can I spawn?

Waaay, waaay more (hundreds or thousands of times) than you would want to spawn for optimal throughput.

The per-thread limitations (on Windows) I'm aware of are:

• 16-bit thread ID
• 4-8KB allocation for user-space stack (typically, much more)
• Non-pageable kernel-space context and stack, something like 16KB

Dotnet probably adds a bunch of per-thread overhead for its own stuff. GC and the like.

One formula I like use for a WAG is:

threads = 2 * (cpu cores + active disk spindles)

The optimal number is usually within a factor of two of that. of that. The theory is that needed threads are proprotional to cpu cores (for obvious reasons), but also that some threads will block on disk I/O. Multiplying by two gives the CPU something to do while other threads are blocked.

Anyway, start with that and measure it. The number of worker threads is the easiest part of the whole problem to adjust later, so don't worry about it too much now.

Answer 4

Just wondered what limit would be hit first by starting many threads. I wrote the following simple test program and tried it. Now I assume that memory is the limiting factor. I was able to run 1000 threads, but without the Thread.Sleep() the system became "a bit inresponsive". With 2000 threads I got an out of memory exception after starting around 1800 threads. (Notebook with a Intel Core 2 Duo T5800 2.0 GHz, 3.0 GiB RAM and a "few" applications running on WIndows XP SP3 with .NET Framework 3.5 SP1)

UPDATE

The out of memory exception is caused by the stack of the threads. After specifying the stack size on the thread constructor (I used 64 kB but got probably the minimum size I don't know at the moment) I was able to start 3500 threads (with Thread.Sleep()).

using System;
using System.Linq;
using System.Threading;

namespace GeneralTestApplication
{
    class Program
    {
        private static void Main()
        {
            Console.WriteLine("Enter the number of threads to start.");

            while (!Int32.TryParse(Console.ReadLine(), out Program.numberThreads)) { }

            Program.counters = new Int64[Program.numberThreads];

            Console.WriteLine("Starting {0} threads.", Program.numberThreads);

            for (Int32 threadNumber = 0; threadNumber < Program.numberThreads; threadNumber++)
            {
                new Thread(Program.ThreadMethod).Start(threadNumber);
            }

            Console.WriteLine("Press enter to perform work on all threads.");
            Console.ReadLine();

            Program.manualResetEvent.Set();

            Console.WriteLine("Press enter to stop all threads.");
            Console.ReadLine();

            Program.stop = true;

            Console.WriteLine("At least {0} threads ran.", Program.counters.Count(c => c > 0));

            Console.ReadLine();
        }

        private static Int32 numberThreads = 0;
        private static Int64[] counters = null;
        private static readonly ManualResetEvent manualResetEvent = new ManualResetEvent(false);
        private static volatile Boolean stop = false;

        public static void ThreadMethod(Object argument)
        {
            Int32 threadNumber = (Int32)argument;

            Program.manualResetEvent.WaitOne();

            while (!Program.stop)
            {
                Program.counters[threadNumber]++;

                // Uncomment to simulate heavy work.
                Thread.Sleep(10);
            }
        }
    }
}

Answer 5

You have to measure. That said, with N cores I usually get the best results by spawning between N+1 and 2N threads. But you have to measure.