How can I consume Firebug net panel data programmatically?

Question

My agile team will be adding new features to a existing realty website. As we add the features we want to have a better handle on the site's overall performance as well as the performance of particular pages.

I would like to automate the gathering of performance metrics on a request/response basis for each page (e.g. what sub requests are sent out by the browser, how many are there, how much data is transferred, and how long did each request take to fulfill).

Firebug currently captures this information in its net panel, however, I haven't found any way to programmatically pull this information out.

Does anyone know of a way to pull this information out after a page has loaded?

We are currently running our user acceptance tests with Selenium and I have considered adding this feature to the selenium interface so that our tests could run and collect the data without starting any other service.

All suggestions are welcome, including ones that leverage other tools/methods to gather the performance metrics.

Thank you.

Answer 1

Okay I admit this is not a direct answer but how about going right to the source? Cut out FireBug and go to the web server. Can the server log events with sufficient granularity to allow calculation of the information you require? Parsing the log file into useful data should not be particularly difficult and has the advantage of being user-platform independent and has the potential to log a greater set of data than that offered by FireBug (Awesome tool btw).

Answer 2

There is a way to use ySlow to beacon out performance data to a URL of your choice. It's not well documented, the only info I found was here:

http://tech.groups.yahoo.com/group/exceptional-performance/messages/490?threaded=1&m=e&var=1&tidx=1

Aside from that I would look into writing a Firebug plugin, I think you can access most Firebug properties. Here's a tutorial: http://www.firephp.org/Reference/Developers/ExtendingFirebug.htm

Answer 3

Ben, I've done this by extended Selenium RC's ProxyHandler to queue up the URLs seen and then allow you to pull them down via some other API. It requires that you proxy everything, which isn't the default behavior of Selenium. The nice thing is that Selenium ends up being both the place to drive automation and collect the results seen.

This is probably a feature we'll soon add to Selenium RC right after we get 1.0 out the door (we're very close!).

Answer 4

Jan Odvarko has written a Tutorial on how to use the new listener functionality within Firebug to log net panel results:

"Since Firebug 1.4a13 the Net panel introduces, among other things, several new events that allow to easily collect all network requests and also related info gathered and computed by Firebug.

This functionality should be useful also in cases where Firebug extensions want to store network activity info into a local database or send it back to the server for further analysis (I am thinking about performance statistics here)."