I need to ensure that any number of websites and services haven't bitten the dust. At the moment I rely on a simple 5 minute cronjob which which uses curl and a file of urls to check that all the responses are correct (200 or rarely 301).
It all works on a basic level, I'm interested in more sophisticated open-source tools (statistics, response times, pretty graphs) - what do you use?
I haven't used it myself, but I'm pretty sure I've heard Nagios spoken of in fairly glowing terms. Worth a try?
Speaking from experience, I'd double the suggestion of using Nagios. I don't think I have found a better open source tool than Nagios that achieves the same stuff.
In my experience nagios works pretty well but seems to be hard to take care of at times. If you want to have stats and everything else on top of what nagios gives you then add cacti. If you want it all in one package you should look into OpenNMS. To get the full benefit of all of these you should be using SNMP too so you can see the details of memory, cpu, disk use as well.
I would recommend the ManageEngine from AdventNet (http://manageengine.adventnet.com/). It is free for up to 5 monitored resources iirc. Outside the free version it is rather cheap.
Out of the box It knows how to monitor webservices, webservers (of varying types, such as IIS, Tomcat, Websphere (5 and 6), and a host of other things such as natively pulling performance data from MSSQL and DB2. Webservices can be monitored just by "http-are-you-there" but you can also specify actual content to send from the wsdl specification, so you know that not only does the other side respond, but it is able to process records as well. (ofcourse this requires you to have a way of deleting these keep-alive data records afterwards but thats a different issue).
For each of the monitored systems you can setup thresholds to the alert, so it has to come 5 times in a row before triggering an SMS alert or automatically restart the service.
As an added bonus it has the best SNMP support I have seen anywhere.
Link to free download : http://manageengine.adventnet.com/products/applications_manager/download-free.html
Linkg to demo: http://demo.appmanager.com/LoginPage.do
(Note: i'm not affiliated with the company)
Zabbix 1.6 supports multiple flows for web monitoring, and will allow you to monitor other things besides that. It's free and very flexible.