Win32: Determine whether stdout handle is char or wchar stream

Question

I'm writing a win32 utility function for our product that needs to call an arbitrary program via the shell and log its output. We do this by redirecting the stdout from the child process into a pipe:

    saAttr.nLength = sizeof(SECURITY_ATTRIBUTES); 
    saAttr.bInheritHandle = TRUE; 
    saAttr.lpSecurityDescriptor = NULL; 

    CreatePipe(&hReadPipe, &hWritePipe, &saAttr, 0);

    // Redirect the first process stdout to our write pipe
    // so that we can read its output from the read pipe.
    startUpInfo.dwFlags = STARTF_USESTDHANDLES;
    startUpInfo.hStdInput = GetStdHandle(STD_INPUT_HANDLE);
    startUpInfo.hStdOutput = hWritePipe;
    startUpInfo.hStdError = GetStdHandle(STD_ERROR_HANDLE);

    CreateProcessA(NULL, szCmdLine, NULL, NULL, TRUE, NORMAL_PRIORITY_CLASS, NULL, NULL, &startUpInfo[i], &procInfo);

There are a wide variety of programs that may be called this way, many of which are not under our control. Currently we're seeing a problem where the output from many programs appears to be truncated after the first character---usually a sure sign that a WCHAR string is mistakenly being used as a CHAR.

How can I tell if the child process is writing to its stdout pipe as a CHAR or a WCHAR?

Answer 1

I think you have to negotiate that when you connect. You could use PeekNamedPipe() and guess based on if it looks like a unicode character...

Answer 2

The bottom line is there is no 100% reliable way to do this. The input/output handles of a process are not encoding specific. They simply operate on a stream of bytes. It's completely possible for a process to write ASCII for awhile and switch to Unicode later on.

Unfortunately it is not possible to tell from a stream of bytes with 100% accuracy what the underlying encoding is. The only way to determine this is to have a hand shake protocol where the process tells you what encoding it will use. Likely not an option in this case.

Links on the subject

• http://stackoverflow.com/questions/90838/how-can-i-detect-the-encodingcodepage-of-a-text-file
• http://blogs.msdn.com/oldnewthing/archive/2004/03/24/95235.aspx