How to get load times of different resources (JS/CSS/Images) on a webpage (on a server)?

Question

Given a URL, on a server (preferably Linux), I'm interested in finding out load times of all different resources loaded by that URL. These resources would be JavaScript/CSS/Images, etc. (I can't simply scrape a page because many of these resources are loaded by JavaScript, say using document.write) Basically, I want to get the information that Firebug's NET tab provides in a machine readable format on a server. How can I do that?

Some of the existing services that provide such functionality are:

• http://www.webpagetest.org/
• http://tools.pingdom.com/
• http://www.site-perf.com/
• http://loadimpact.com/

How do these services work? I was exploring a Java library called HTMLUnit which can execute JavaScript on a page on server-side but I'm not sure if it can provide the information I am looking for.

Answer 1

They (or at least this one) use tools like iMacros for Firefox on the backend. Together with the Firebug extension this gives them exactly the numbers you need. In any case, the only way to get accurate numbers is to load the website in some real browser (headless browsers will fail with complex websites).