Asynchronous I/O with Named Pipes WinAPI

Question

Ok, I have asked a few questions about different facets of trying to accomplish what I want to do. This time I'm having big issues just reading from a named pipe. I think I have harvested enough information to possibly complete the project I am working on if I can set this up properly. I will include all relevant code below, but my mission is this: read the output (continuously) from a program I did not write and post it to the WinAPI. So my problem is that I have just switched from anonymous pipes to named pipes and I am having issues trying to properly set them up so I can retrieve information. I have a framework setup based off of an example from the MSDN.

#define WAIT_TIME 2 // 2s
#define INSTANCES 4 // Number of threads
#define CONNECT_STATE 0
#define READ_STATE 1
#define WRITE_STATE 2
#define WORLDRD 0
#define WORLDWR 1
#define WORLDINRD 2
#define WORLDINWR 3
#define BUFSIZE 0x1000 // Buffer size 4096 (in bytes)
#define PIPE_TIMEOUT 0x1388 // Timeout 5000 (in ms)

void Arc_Redirect::createProcesses()
{
TCHAR programName[]=TEXT("EXEC_PROGRAM.exe");
PROCESS_INFORMATION pi; 
STARTUPINFO si;
BOOL bSuccess = FALSE; 

ZeroMemory(hEvents,(sizeof(hEvents)*INSTANCES));
ZeroMemory(outStd,(sizeof(PIPE_HANDLES)*INSTANCES));

// Prep pipes
for(int i=0;i<INSTANCES;i++)
{
    hEvents[i] = ::CreateEvent(NULL, TRUE, FALSE, NULL);

    if(hEvents[i] == NULL)
        throw "Could not init program!";

    outStd[i].o1.hEvent = hEvents[i];

    outStd[i].hPipeInst = ::CreateNamedPipe(
        TEXT("\\\\.\\pipe\\arcworld"), PIPE_ACCESS_DUPLEX | FILE_FLAG_OVERLAPPED, PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT,
        PIPE_UNLIMITED_INSTANCES, BUFSIZE*sizeof(TCHAR), BUFSIZE*sizeof(TCHAR), PIPE_TIMEOUT, NULL);

    if(outStd[i].hPipeInst == INVALID_HANDLE_VALUE)
        throw "Could not init program!";

    outStd[i].pendingIO = getState(outStd[i].hPipeInst,&outStd[i].o1);

    outStd[i].dwState = outStd[i].pendingIO ?
        CONNECT_STATE : READ_STATE;
}

// Set stuff up
ZeroMemory( &pi, sizeof(PROCESS_INFORMATION));
ZeroMemory( &si, sizeof(STARTUPINFO));
si.cb = sizeof(STARTUPINFO); 
si.hStdError = outStd[WORLDRD].hPipeInst;
si.hStdOutput = outStd[WORLDRD].hPipeInst;
si.hStdInput = outStd[WORLDINWR].hPipeInst;
si.dwFlags |= STARTF_USESHOWWINDOW|STARTF_USESTDHANDLES|FILE_FLAG_OVERLAPPED;

    // Start our process with the si info
CreateProcess(programName,NULL,NULL,NULL,FALSE,0,NULL,NULL,&si,&pi);
}

BOOL Arc_Redirect::getState(HANDLE hPipe, LPOVERLAPPED lpo)
{
    BOOL connected, pendingIO = FALSE;

    // Overlap connection for this pipe
    connected = ::ConnectNamedPipe(hPipe,lpo);

    if(connected)
        throw "ConnectNamedPipe(); failed!";

    switch(GetLastError())
    {
        case ERROR_IO_PENDING:
            pendingIO = TRUE;
        break;
        case ERROR_PIPE_CONNECTED:
            if(SetEvent(lpo->hEvent))
                break;  
        default:
            throw "ConnectNamedPipe(); failed!";
        break;
    }

    return pendingIO;
}

outStd[INSTANCES] is defined as PIPE_HANDLES (a custom struct) which is below

typedef struct
{
    HANDLE hPipeInst;
    OVERLAPPED o1;
    TCHAR chReq[BUFSIZE];
    TCHAR chReply[BUFSIZE];
    DWORD dwRead;
    DWORD dwWritten;
    DWORD dwState;
    DWORD cbRet;
    BOOL pendingIO;
} PIPE_HANDLES, *LPSTDPIPE;

Now from here is where I am getting a bit lost. I'm not sure where to go. I tried using the loop in the MSDN example, but it didn't work properly for what I am looking to do. I need to take the read end of the pipe and retrieve the information (again, continuously) while having a write end opened as well for whenever I may need to write to it. Does anyone have any ideas? I have been trying to do a ReadFile() as I would do with an anonymous pipe, but it does not appear to be working in the same fashion.

Regards,
Dennis M.

Also, please note: The code is a bit sloppy because I have been working with it, so I apologize. I will definitely be cleaning it up after I get this to function properly.

Answer 1

In the linux world, one program can write to a named pipe through write/fwrite calls and another program can read it through read/fread().

The FULL path of the named pipe must be used in read/write operations.

Answer 2

Have you tried passing PIPE_NOWAIT instead of PIPE_WAIT in the CreateNamedPipe call? This will allow ReadFile and WriteFile to be non-blocking.

Alternatively, have you tried using async IO? You're passing the FILE_FLAG_OVERLAPPED flag, so this should work. If you've tried it, what problems did you encounter?

Answer 3

You should have two OVERLAPPED structures, one for reading and one for writing. Also you need one event handle per pipe for when you want to close the pipe, and one more event when you want to abort all (and close application). You can have one WaitForMultipleObjects for every operation that all pipes are currently involved with, or separate reading from writing in two threads with one WFMO in each. I would go with only one thread, because closing the pipe is then simpler (otherwise you need to have some reference counting on the pipe handle and close it only when reference count drops to zero).

When you get one event then process it, and try WFMO with 0 seconds on all handles that were in the array after the one you just processed. This way no pipe will get starved. When 0 second WFMO elapses repeat normal WFMO from beginning.

If you need high concurrency then process events in separate threads, and omit the currently processing handles from WFMO. However, tracking all the handles then gets a little complicated.