Does it make sense to spawn more than one thread per processor?

Question

From a logical point of view an application may need dozens or hundreds of threads, some of which will we sleeping most of the time, but a very few will be always running concurrently. The question is: Does it make any sense to spawn more concurrent threads than processors there are in a system, or it is a waste?

I've seen some server applications that implement a scheduler to logically manage tasks (often called jobs), but also spawning a lot of threads, so I don't see where the benefit is.

Thanks in advance.

Answer 1

Everytime you have a task waiting for an I/O operation it does make a sense to enclose it in a thread and fire it up. There's a great probability that your thread will be suspended while waiting for the I/O operation to finish. When it gets waken up the result will be waiting for it.

Answer 2

Because all modern OS are multi tasking: each thread gets a time share from the processor. It is not actually concurrent execution but given the processor can handle thousands of requests per sec, it is an "apparent" concurrent execution.

So yes, if the case needs, it does make sense to multi-thread on a single processor.

Answer 3

One of the benefits is when you upgrade your hardware, which will likely get more processors/cores.

Answer 4

Short answer is "yes".

Even thought you could gain more from multithreading on a multiprocessor environement, it's still a useful technology on a single processor machine, mainly because it means you'll delegate some work to the process scheduler, who should have much better information than you have.

If you don't multithread, you'll end up doing the scheduling job yourself, which could be a good thing if that's what you need, but will most probably be both tedious and inefficient

Answer 5

This may make sense if

1. your program design benefits, in that you have parallel tasks that are best implemented in threads, or

2. some of your threads are I/O-bound, so they do not utilize the processors /cores on their own.

Answer 6

Sure. If your software makes frequent use of disk or network IO, you can often improve throughput by adding a few more threads. Those extra threads will be awake and doing stuff while the other threads are blocking on IO.

Answer 7

Other have talked about situations in which it almost certainly does make sense (when you are doing any kind of slow IO).

It might not be a good idea if:

• your threads are doing CPU bound work

and

• the threads each want to use a lot (i.e. significant compared to the cache size) of memory that does not overlap

In this case there is the possibility of causing unnecessary cache misses.

Answer 8

I found that when writing data parsers which handle larger sets of data over a network it is best to create a thread for each letter of the alphabet (pertaining to the data) and get the program to be more CPU and memory bound. The I/O boundedness inherit with network and disk operations is a major bottleneck so you may as well "get started" on the other data files instead of doing the work sequentially.

On a quad core, it would certainly make sense to start more than four threads. It is unlikely that those 4 threads would be spread across more than one of the cores, especially with todays processor speeds.

Answer 9

According to Herb Sutter (one of the leading experts on concurrency), one of the Pillars of Concurrency is Responsiveness and Isolation Via Asynchronous Agents. The summary is:

Stay responsive by running tasks independently and tasks asynchronously, communicating via messages.

Great article (and the series as a whole!). I am still waiting for the book.