Cross-platform (linux/Win32) nonblocking C++ IO on stdin/stdout/stderr

Question

I'm trying to find the best solution for nonblocking IO via stdin/stdout with the following characteristics:

• As long as there is enough data, read in n-sized chunks.
• If there's not enough data, read in a partial chunk.
• If there is no data available, block until there is some (even though it may be smaller than n).

The goal is to allow efficient transfer for large datasets while processing 'control' codes immediately (instead of having them linger in some partially-filled buffer somewhere).

I know I can achieve this by using threads and a istream::get() loop, or by writing a bunch of platform-specific code (since you can't select() on file handles in windows)... ((There is also istream::readsome() which seems promising, but the only results I can find on google were of people saying it doesn't actually work well.))

Since I haven't done much coding w/ these APIs, perhaps there is a better way.

Answer 1

Maybe boost::asio can be of use for you?

Answer 2

I used the threads and platform specific code. See my answer to another question. I was able to put the OS-specific stuff in inputAvailable() (Linux uses select, Windows just returns true). I could then use WaitForSingleObject() with a timeout on Windows to try to let the thread complete, then TerminateThread() to kill it. Very ugly, but the team didn't want to use this bit of boost.

Answer 3

I did something similar to jwhitlock ... I ended up with a StdinDataIO class that wraps around the appropriate OS-specific implementation(*) so that the rest of my program can select() on the file descriptor StdinDataIO provides, remaining blissfully ignorant of Windows' limitations regarding stdin. Have a look here and here if you like, the code is all open-source/BSD-licensed.

(*) the implementation is a simple pass-through for Linux/MacOSX, and in Windows it's a rather complex process of setting up a child thread to read from stdin and send the data it receives over a socket back to the main thread... not very elegant, but it works.