can we write a program to measure how much traffic any website has like Alexa does?

Question

Can we write a program to measure how much traffic any website has like Alexa or ComScore does? Do we need to be in the middle of an internet backbone or have access to such traffic data? can we write any program to measure it just using a consumer-level internet connection?

Answer 1

You can't do that, no. Alexa does it by giving away a toolbar that people install into their browsers, that reports back to Alexa which websites they visit.

Even being "in the middle of an internet backbone" wouldn't help much, because people will visit the site from all kinds of different routes. You'd need control over the servers from which the website is served.

Answer 2

There are a a number of methods employed, some are better than others. As mentioned Alexa has a toolbar that is used by a subset of the internet user community. This has intrinsic bias associated with it - people who know about alexa and use the toolbar.

Other firms like Quantcast look at users cookies and have collaborative programs with sites to provide essentially ratings #s for the sites.

Answer 3

The way GoogleAnalytics uses is probably the easiest to do, technically: have a script on each site that will report back to you on page access.

The downside to this: you need the sites' participation - the site itself has to embed your tracking code (that's the urchin.js script et al).

Answer 4

If you want to measure traffic you need a way of counting the number of times a specific website is used by end users.

There are basically 5 places to do make measurements:

1. Modify the website to retrieve 'something extra' and count that (i.e. the way omniture and the likes work).
2. Change something in the browser (like the plugin Alexa uses).
3. Wiretap the network (at ISP level)
4. Wiretap the network (at application hosting provider level)
5. Change the server application and let it log the usage itself.

In all but the last two options you're not measuring at all. You're measuring something for which you assume some kind of relation ship with what really happens. So in reality you're reporting more on your assumptions then on your measurements.

• Omniture assumes that it counts all visitors... Except the ones who have "addblock plus", or a browser that is a bit too old for the javascript, or are google bots, or are deliberately trying to overload a site ... etc.
• Alexa assumes that the users of their plugin are representative for the 'whole' internet. Sure ....

The only reliable way of doing website statistics is by making technical changes at the actual location where those websites are hosted. And I'm convinced Alexa won't let you do that.

So "Can you write" such a program? Sure you can. The 'only' issue is getting it deployed in a place where you can get useful numbers.

Answer 5

If you have a team of good programmers, you probably can write a program, because Alexa and ComScore did it.

You can't count traffic for any site you like, not sitting in the middle of backbone, nor from home. You need to sit in front of every site you like to count, or in front of every visitor of that site.