IOCP in custom thread pool

Question

I'm currently searching the internet for a custom thread pool implementation. I found an implementation which uses IOCP's. I'm wondering what the benefit is, of using them? Do they provide work stealing, or something like that, I could really find an answer...

Answer 1

IOCP = "IO Completion Port". It is a kernel object built into the Windows OS that is there to give you an intelligent way to manage multithreaded asynchronous IO.

In very simplistic (and a little over-simplified) terms, you tell the IOCP about the IO jobs you want done. It will perform them asynchronously and maintain a queue of the results of each of those jobs. Your call to tell the IOCP about the job returns immediately (it does not block while the IO happens). You are returned an object that is conceptually like the .NET IAsyncResult ... it lets you block if you choose to, or you can provide a callback, or you can periodically poll to see if the job is complete.

While doing those jobs, the IOCP uses a thread pool. The thread pool tries to limit the number of threads to the number of processors or less (that's configurable, but the intent and the default is to limit it to the number of processors). However, the IOCP makes a provision for the fact that sometimes, the tasks on these threads might block. Asycn IO tasks won't block, but you might have provided it some other kind of task. So, you can give the IOCP another number ... this is a number of threads above the "usual maximum", which the IOCP is allowed to go up to due to one of the other threads being blocked. The goal is to have up to the "usual maximum" of threads that are actually doing something (ie, not blocked). If this happens, then for a while the IOCP will be using more threads than the usual max, but it will refuse to make any new threads until it can get back down to the "usual max".

This is a brief summary, conceptual only, and as I said, over-simplified in some ways. But it should give you the general idea. Jeffrey Richter's books on the Windows OS cover this in detail (but these books are out of print now). You can find these books used, and they actually cost more used than they did originally. I think "Advanced Windows" is the title you'd want, but there may have been an updated version of the book with a different title.

Answer 2

You can use an IOCP implementation if you plan to use Windows IO functions from within your worker threads. IO function could be read/write to disk or to network.

Check Push Framework http://www.pushframework.com to see a usage of IOCP for real time server implementation.