I didn't find anything out there that fit my needs, so I learned a little Powershell scripting and rolled a solution that should be useful to others as well. Assuming a Windows platform (otherwise use monit!), Powershell is really powerful and easy.
sample-monitor.ps1 script:
$webClient = new-object System.Net.WebClient
###################################################
# BEGIN USER-EDITABLE VARIABLES
# the URL to ping
$HeartbeatUrl = "http://someplace.com/somepage/"
# the response string to look for that indicates things are working ok
$SuccessResponseString = "Some Text"
# the name of the windows service to restart (the service name, not the display name)
$ServiceName = "Tomcat6"
# the log file used for monitoring output
$LogFile = "c:\temp\heartbeat.log"
# used to indicate that the service has failed since the last time we checked.
$FailureLogFile = "c:\temp\failure.log"
# END USER-EDITABLE VARIABLES
###################################################
# create the log file if it doesn't already exist.
if (!(Test-Path $LogFile)) {
New-Item $LogFile -type file
}
$startTime = get-date
$output = $webClient.DownloadString($HeartbeatUrl)
$endTime = get-date
if ($output -like "*" + $SuccessResponseString + "*") {
# uncomment the below line if you want positive confirmation
#"Success`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds" >> $LogFile
# remove the FailureLog if it exists to indicate we're in good shape.
if (Test-Path $FailureLogFile) {
Remove-Item $FailureLogFile
}
}
else {
"Fail`t`t" + $startTime.DateTime + "`t`t" + ($endTime - $startTime).TotalSeconds + " seconds" >> $LogFile
# restart the service if this is the first time it's failed since the last successful check.
if (!(Test-Path $FailureLogFile)) {
New-Item $FailureLogFile -type file
"Initial failure:" + $startTime.DateTime >> $FailureLogFile
Restart-Service $ServiceName
}
}
The only logic in this script is that it will only try to restart the service once after an initial failure. This is to prevent a situation where a service takes a while to restart, and while it's restarting, the monitor keeps seeing the failure and restarts again (bad infinite loop). Otherwise you can do just about anything, like add email notifications, or do more than just restart a service.
This script will execute once, which means you'll need to control its repetition externally. You could put it in an infinite loop right in the script, but that seems a little flaky. I used windows Task Scheduler, executing it like so:
Program: Powershell.exe
arguments: -command "C:\projects\foo\scripts\monitor.ps1" -noprofile
Start In: C:\projects\foo\scripts
You could also use a more robust scheduler like VisualCron, plug it into a windows service, or via an application server scheduler like Quart.NET. In my case the task scheduler works fine.