Is there a buffer size attached to stdout?

Question

I am trying to find some information on data limits related to stdout on Windows. I can't seem to find the information on MSDN.

1. Is there a limit to how much data can be written to stdout? If so, what happens if the limit is reached? Is the data lost?

2. If stdout is redirected (for example, by launching the process from .Net and using the ProcessStartInfo.RedirectStandardOutput property), does that have any effect on how much data can be written? As I read from the stdout stream in the calling process, does that affect the limitations?

3. Are these limits related in any way to named pipes?

Answer 1

It depends where it's going - but yes, if you redirect the output in .NET you can easily run into problems if you don't read the output. When the buffer runs out, writes to stdout in the child process will block. One common-ish cause of deadlock is a "parent" process waiting for the "child" to exit, and then reading the output - that won't work if the child needs the parent to read the output to free up buffer space.

.NET has made this slightly easier by allowing an event-driven approach with Process.OutputDataReceived and Process.ErrorDataReceived. This means you don't need to start two threads (one to read stdout, one to read stderr) just to keep the process from blocking...

Answer 2

Some things to keep in mind:

1) Jon is right - if the buffer limit is reached, the write call in your subprocess will block. You need to drain the stdout stream if it is not being redirected somewhere that will cause it to automatically drain - like a file. Pipes need to be drained, and usually, if you can "attach" to a subprocess' output, you're attaching to a pipe.

2) The I/O to an output stream is probably buffered, which means that if the subprocess writes some information to stdout without explicitly calling flush(), which is almost always the case, you might not see the output. Flush is automatically called when the process exits, so if it's a short small subprocess you should be OK, but if it's not, you have no real way of forcing its output to show up when you want it to.

3) Named pipes are essentially a buffer that the OS maintains that can be written to and read from - that is, they're like a file you can write to from one process and read from in another, without actually having the overhead of having a file on disk. Very handy for communication between processes, but all the I/O limitations with buffering / full buffers still apply.

Answer 3

What are the limits? Where is that documented?