How to do event driven programming between 2 separate C# programs?

Answer 1

I would recommend MSMQ. :) I am not sure why you are not using it, however you did mention it in your question. This is pretty much exactly what MSMQ was designed for...durable eventing between applications. It primarily supports the pub/sub message model...which based on your "Ideal Solution" is exactly what you need: TC is the publisher, TP is a subscriber. TC registers the task, then drops a message in its publish queue. The TP task does not need to be up and running for TC to successfully drop a message in its queue, however when the TP task IS running, it will receive notifications and handle messages in the queue in the prioritized order the arrive.

If MSMQ is not an option, you could also use WCF. Rather than pub/sub, with WCF you could opt for a FAF (fire and forget) message model. TP would publish a service which TC would consume. TC would only need to fire a message off to TP's service to notify TP of new tasks. The downside of this model is that TC is dependent on TP, which might be less than idea. TP also has to be running for TC to successfully function, since it is dependent upon TP's service. With the MSMQ approach, neither TP nor TC are dependent on each other, they are only dependent upon MSMQ (a lower-coupling approach.)

EDIT:

An example of how to use MSMQ to fire events from TC and respond to events in TP.

// TC message queue manager, sends messages
public class TaskMessageQueueManager
{
  public void NotifySubscribersOfNewTasks()
  {
    var queue = getQueue(".\private$\TaskNotifications");
    queue.Send("Tasks waiting.");
  }

  private MessageQueue getQueue(string name)
  {
    MessageQueue queue = null;
    try
    {
      if (!MessageQueue.Exists(name))
      {
        queue = MessageQueue.Create(name);
      }
      else
      {
        queue = new MessageQueue(name);
      }
    } 
    catch (Exception ex)
    {
      throw new InvalidOperationException("An error occurred while retrieving the message queue '" + name + "'.", ex);
    }

    return queue;
  }
}

// TP message queue handler, receives messages
public class TaskMessageQueueHandler
{
  private Thread m_thread;
  private ManualResetEvent m_signal;

  public void Start()
  {
    m_signal = new ManualResetEvent(false);
    m_thread = new Thread(MSMQReceiveLoop);
    m_thread.Start();

  }

  public void Stop()
  {
    m_signal.Set();
  }

  private void MSMQReceiveLoop()
  {
    bool running = true;
    MessageQueue queue = getQueue(".\private$\TaskNotifications");

    while (running)
    {
      try
      {
        var message = queue.Receive(); // Blocks here until a message is received by MSMQ

        if (message.Body.ToString() == "Tasks waiting.")
        {
          // TODO: Fire off process, perhaps another thread, to handle waiting tasks
        }

        if (m_signal.WaitOne(10)) // Non-blocking check for exit signal
        {
          running = false; // If Stop method has been called, the signal will be set and we can end loop
        } 
      }
      catch
      {
         // handle error
         running = false;
      }
    }
  }
}

The message does not have to be simple text. You can send an object or object graph, and it will automatically be serialized and formatted as XML by default. I believe you can also serialize data in a binary format, if that is what you need. Either way, you'll notice that there are no Thread.Sleep calls or polling anywhere. The loop exits based on a ManualResetEvent, allowing you to cleanly end the thread without a hard abort.

Answer 2

Check out SQL Server 2005 Query Notifications. I just learned that it seems that it's been pulled from SQL Server 2008 so it might not be the best idea after all.

I second that MSMQ is probably a much better way. Combined with nServiceBus you could have a winner.

Another approach could be thread synchronization events.

In TP you could have something like:

EventWaitHandle taskEvent = new EventWaitHandle(true,
                EventResetMode.AutoReset,
                "newTask",
                out wasCreated);
new Thread(WaitForTask).Start();
...

public void WaitForTask() { while (true) { taskEvent.WaitOne(); ProcessTasks();} }

And in TC:

bool eventExist;
while (!eventExist)
{
    try
    {
        taskEvent= EventWaitHandle.OpenExisting("newTask");
        eventExist = true;
    }
    catch (WaitHandleCannotBeOpenedException)
    {
        eventExist = false;
        Thread.Sleep(1000);
    }
}

CreateNewTask();
taskEvent.Set();

Not that I see how a call every five second could be such a performance hog.

Answer 3

When there’s no new tasks it’s a bit of waste to fire up a select from where SQL statement.

A select on a single table, every 5 secs with simple criteria and none rows returned usually costs next to nothing, dont worry.