How to keep asynchronous parallel program code manageable (for example in C++)

Question

I am currently working on a server application that needs to control a collection devices over a network. Because of this, we need to do a lot of parallel programming. Over time, I have learned that there are three approaches to communication between processing entities (threads/processes/applications). Regrettably, all three approaches have their disadvantages.

A) You can make a synchronous request (a synchronous function call). In this case, the caller waits until the function is processed and the response has been received. For example:

const bool convertedSuccessfully = Sync_ConvertMovie(params);

The problem is that the caller is idling. Sometimes this is just not an option. For example, if the call was made by the user interface thread, it will seem like the application has blocked until the response arrives, which can take a long time.

B) You can make an asynchronous request and wait for a callback to be made. The client code can continue with whatever needs to be done.

Async_ConvertMovie(params, TheFunctionToCallWhenTheResponseArrives);

This solution has the big disadvantange that the callback function necessarily runs in a separate thread. The problem is now that it is hard to get the response back to the caller. For example, you have clicked a button in a dialog, which called a service asynchronlously, but the dialog has been long closed when the callback arrives.

void TheFunctionToCallWhenTheResponseArrives()
{
    //Difficulty 1: how to get to the dialog instance?
    //Difficulty 2: how to guarantee in a thread-safe manner that
    //              the dialog instance is still valid?
}

This in itself is not that big a problem. However, when you want to make more than one of such calls, and they all depend on the response of the previous one, this becomes in my experience unmanageably complex.

C) The last option I see is to make an asynchronous request and keep polling until the response has arrived. In between the has-the-response-arrived-yet checks, you can do something useful. This is the best solution I know of to solve the case in which there is a sequence of asynchronous function calls to make. This is because it has the big advantage that you still have the whole caller context around when the response arrives. Also, the logical sequence of the calls remains reasonably clear. For example:

const CallHandle c1 = Sync_ConvertMovie(sourceFile, destFile);
while(!c1.ResponseHasArrived())
{
    //... do something in the meanwhile
}
if (!c1.IsSuccessful())
    return;

const CallHandle c2 = Sync_CopyFile(destFile, otherLocation);
while(!c1.ResponseHasArrived())
{
    //... do something in the meanwhile
}
if (c1.IsSuccessful())
    //show a success dialog

The problem with this third solution is that you cannot return from the caller's function. This makes it unsuitable if the work you want to do in between has nothing to do at all with the work you are getting done asynchronously. For a long time I am wondering if there is some other possibility to call functions asynchronously, one that doesn't have the downsides of the options listed above. Does anyone have an idea, some clever trick perhaps?

Note: the example given is C++-like pseudocode. However, I think this question equally applies to C# and Java, and probably a lot of other languages.

Answer 1

I tend to go with B, but instead of calling forth and back, I'd do the entire processing including follow-ups on a separate thread. The main thread can meanwhile update the GUI and either actively wait for the thread to complete (i.e. show a dialog with a progress bar), or just let it do its thing in the background and pick up the notification when it's done. No complexity problems so far, since the entire processing is actually synchronous from the processing thread's point of view. From the GUI's point of view, it's asynchronous.

Adding to that, in .NET it's no problem to switch to the GUI thread. The BackgroundWorker class and the ThreadPool make this easy as well (I used the ThreadPool, if I remember correctly). In Qt, for example, to stay with C++, it's quite easy as well.

I used this approach on our last major application and am very pleased with it.

Answer 2

You could consider an explicit "event loop" or "message loop", not too different from classic approaches such as a select loop for asynchronous network tasks or a message loop for a windowing system. Events that arrive may be dispatched to a callback when appropriate, such as in your example B, but they may also in some cases be tracked differently, for example to cause transactions in a finite state machine. A FSM is a fine way to manage the complexity of an interaction along a protocol that requires many steps, after all!

One approach to systematize these consideration starts with the Reactor design pattern.

Schmidt's ACE body of work is a good starting point for these issues, if you come from a C++ background; Twisted is also quite worthwhile, from a Python background; and I'm sure that similar frameworks and sets of whitepapers exist for, as you say, "a lot of other languages" (the Wikipedia URL I gave does point at Reactor implementations for other languages, besides ACE and Twisted).

Answer 3

Like Alex said, look at Proactor and Reactor as documented by Doug Schmidt in Patterns of Software Architecture.

There are concrete implementations of these for different platforms in ACE.