how to determine that one instance of a service is down and launch a new instance

Question

Hey I have the following situation: a WinService which is processing some stuff from a queue, writing to other tables (during processing) and after it finishes it deletes the entry and moves to the next one.

I would need a mechanism to determine if the instance is alive and going and if not to start or wake up a second instance.

What I've thought so far is to have two instances going on where each tries to pick up stuff from the queue and process it (tries- as in see if it can- like no keepAliveFlag is set or lastUpdateDate of a record is too long ago). There could be the situation when they process the same record- this is the disadvantage.

For this above proposed solution, I would also need a sort of mechanism in the case when an instance tries to write a keepAliveFlag and does not succeed to stop.

Aside from this solution, there is also the solution for the instance to constantly write stuff into db and the second instance to check if new stuff is written and if not wake up or launch the second instance. But several problems may appear: what happens if the former master wakes up? And in general this second scenario seems harder to implement correctly.

Now the question: what would be the correct pattern to implement this?

Answer 1

Services are singleton in a Windows system. The only way to support multiple instances would be to install multiple identical binaries with different service names.

You can monitor the status of any given service using the [ServiceController][1] class, using its Status property to determine current state and act accordingly.

Answer 2

What one of my teams does at the moment is this:

• On each machine, install an instance of a monitor (itself a Windows service).
• This monitor reads (from a central database) the list of local services that it should monitor.
• It also reads the working/non-working hours for each local service that it's monitoring.
• Then it monitors each service for regular heartbeats (and other status messages).
• If a local service has not started during its working hours, the monitor starts it.
• If a local service is running but not heartbeating, the monitor restarts it.
• The monitor logs in the database the state of each service and sends info/warning/alert messages as appropriate.