Elegant way to determine total size of website?

Question

is there an elegant way to determine the size of data downloaded from a website -- bearing in mind that not all requests will go to the same domain that you originally visited and that other browsers may in the background be polling at the same time. Ideally i'd like to look at the size of each individual page -- or for a Flash site the total downloaded over time.

I'm looking for some kind of browser plug-in or Fiddler script. I'm not sure Fiddler would work due to the issues pointed out above.

I want to compare sites similar to mine for total filesize - and keep track of my own site also.

Answer 1

Have you tried Firebug for Firefox?

The "Net" panel in Firebug will tell you the size and fetch time of each fetched file, along with the totals.

Answer 2

Firebug and HttpFox are two Firefox plugin that can be used to determine the size of data downloaded from a website for one single page. While Firebug is a great tool for any web developer, HttpFox is a more specialized plugin to analyze HTTP requests / responses (with relative size).

You can install both and try them out, just be sure to disable the one while enabling the other.

If you need a website wide measurement:

• If the website is made of plain HTML and assets (like CSS, images, flash, ...) you can check how big the folder containing the website is on the server (this assumes you can login on the server)
• You can mirror the website locally using wget, curl or some GUI based application like Site Sucker and check how big the folder containing the mirror is
• If you know the website is huge but you don't know how much, you can estimate its size. i.e. www.mygallery.com has 1000 galleries; each gallery has an average of 20 images loaded; every image is stored in 2 different sizes (thumbnail and full size) an average of for _n_kb / image; ...

Keep in mind that if you download / estimating a dynamic websites, you are dealing with what the website produces, not with the real size of the website on the server. A small PHP script can produce tons of HTML.