Limit on cout stream?

Question

A puzzle that hit me. In some simple test harness code, if I stream too many characters to stdout, the program fails. Strange but very reproducable. This may be a Windows only issue, but it's easy to see:

#include <iostream>
#include <deque>

using namespace std;

int main() 
{
  deque<char> d;
  char c;

  while (cin.get(c)) d.push_back(c);

  for (deque<char>::reverse_iterator j = d.rbegin(); j != d.rend(); j++)
    cout << (*j);
}

The previous code just loads a stream of chars from stdin and outputs them in reverse order. It works fine for up to 100K or so characters, but dies with "error writing stdout" message in Windows for files that are larger. It always dies with the same character. A shell command like "cat bigfile.txt | reverse.exe" is all you need to reproduce the problem. Both MSFT and Intel compilers both act similarly.

I realize there may be a buffer on stdout, but shouldn't that be flushed automatically when it's filled?

Answer 1

No such issue here:

C:\Temp> cl
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.21022.08 for 80x86

EDIT: More Info

I tested this by compiling the program you posted. The I created a file consisting of 0123456789 repeated 100,000 times (making its size 1,000,000 bytes). Then, I ran

C:\Temp> t.exe < test.in

as well as

C:\Temp> cat test.in | t.exe

and

C:\Temp> t.exe < test.in > test.out

There were no issues. It did take quite a while to wait for the 1,000,000 characters to scroll by though.

Answer 2

Can you sleep for a short period of time during each loop iteration, or perhaps every 100 iterations? This would give the OS a chance to flush the buffer.

I don't know what the command for doing this is in c++, but in c# it is

System.Threading.Sleep(10);

Answer 3

You may try to force the buffer to flush its content this way:

cout << (*j) << std::flush;

Otherwise std::endl works also, but provide and end of line too (which you don't want I suppose ?)

Answer 4

Thanks for all the suggestions, especially to Michael Burr who correctly theorized that the cat command, not reverse.exe, may be failing! That's exactly what it was.. reverse.exe < bigfile.txt works fine, but cat bigfile.txt | reverse.exe fails with "error writing stdout". Now why CAT would fail is also a mystery but at least it's now not something code related.

Answer 5

I've had this problem before occur if you tried to write a special character to stdout on win32. Any such characters in your test data?

Answer 6

The problem will probably be the pipe operator (|) not "cat". Windows command interpreter[1] doesn't have real pipes (like Unix) and simulates them using temp files. You might be running out of disk space or overflowing some buffer in the command interpreter.

You could try "type bigfile.txt | reverse.exe" and see if you get the same results.

[1] At least older versions didn't have real pipes. I haven't looked at the latest versions. It is interesting that Michael Burr couldn't reproduce it on Vista x64. Maybe MS has fixed the problem.