I assume you understand that both functions crash due to exhaustion of the stack after an attempt at infinite recursion. I think what you are asking is: why would the cout example not crash with "Stack Overflow" also?
I do not think the answer has to do with the compiler's detection of tail recursion. If the compiler optimized the recursion away, neither example should crash.
I have a guess as to what is going on. "Stack Overflow" exception is implemented in some cases (e.g., Windows) with a single Virtual Memory "guard page" allocated at the end of the stack. When a stack access hits this guard page a special exception type is generated.
Since the Intel small-granularity page is 4096 bytes long, the guard page stands guard over a range of memory that size. If a function call allocates more than 4096 bytes of local variables, it is possible that the first stack access from it will actually stretch beyond the guard page. The next page can be expected to be unreserved memory, so an access violation would make sense in that case.
Of course you don't explicitly declare any local variables in your example. I would assume that one of the operator<<() methods allocates more than a page of local variables. In other words, that the Access Violation occurs near the beginning of an operator<<() method or some other part of the cout implementation (temporary object constructors, etc.)
Also, even in the function you wrote, the operator<<() implementations are going to need to create some storage for intermediate results. That storage is probably allocated as local storage by the compiler. I doubt it would add up to 4k in your example, though.
The only way to really understand would be to see a stack trace of the access violation to see what instruction is triggering it.
Got a stack trace of the access violation and a disassembly around the area of the faulting opcode?
If you are using the Microsoft C compiler, another possibility is that printf() and your own function were compiled with /Ge and operator<<() was not, or that only your function was compiled with /Ge and factors similar to those described above coincidentally cause the behavior you see -- because in the printf() example the crash happens while your function is being called and in the operator<<() case while you are calling the library.