Windows Services and shutting down EC2 nodes

Question

I've got a fairly simple Windows service running on testbed EC2 nodes right now that basically:

• Calls a Web Service on a machine in our local data center when the service starts (the service is set to Automatic startup, so it runs when the instance comes online).
• Calls another Web Service once a minute as a monitoring heartbeat.
• Calls a third Web Service when the service stops.

We're using this in our Linux EC2 instances perfectly fine, and the Windows service works fine on startup, heartbeat, and manual stoppage.

The problem is that when terminating the node via ElasticFox, Rightscale, etc. the service never seems to be stopped, or at least, it never makes the Web Service call. Any thoughts?

[Note that this is copied verbatim from my post on the AWS developer forums, but that is super-low traffic compared to SO. Apologies!]

Edit: Regarding the point about the network going down, I'm also not the most well-versed in Windows services, if I set Netman as a service dependency, would the order of service termination be guaranteed to respect that?

Regarding EBS, the model we've adopted for a similar system running on plain ol' Linux instances treats app servers as ephemeral, so EBS wouldn't benefit us that much. We use Rightscale on top of EC2, which makes hooking into Linux instances shutting down, so we just ship off the instance's logs elsewhere.

Edit #2: Well, I got a reply from the AWS staff much faster than I expected:

For windows, terminating through API (which means through elasticfox ,rightscale etc.) is a hard shutdown due to some technical limitations. It means it is like turning off your machine with power button instead of shutdown.

Interesting..

Answer 1

A lot of things happen when Windows shutsdown so my guess is something has already stopped that your web service call needs, first guess would be the network connection.

It's a bit of a tricky one since you can't keep a local log file on the C or D drives to check whether the call is at least ebing attempted since you'll never be able toa ccess them again once the instance has stopped. Do you use an EBS volume with your instances that could hold a log file?

Answer 2

I tested the startup and shutdown group policy scripts with the new Windows EBS root volumes. Both these scripts run as expected during the appropriate sequences when you stop and start these instances. An alternative, therefore, might be to run your instances using EBS root volumes and issue a stop command to ensure the Windows instance executes all shutdown sequences. Once the instance is stopped, you should then be able to terminate the instance and release any EBS volumes that are associated with it.