Why don't client computers serve information to other clients over the internet in order to reduce server bandwidth?

Question

Take an image hosting service for example. In order to reduce bandwidth/the number of times you serve images, is it possible to have a client who has just viewed an image then SERVE that same image file (or pieces of that file) to another client who wishes to view the same page/image? Are there security related issues that prevent this from happening or is my lack of understanding of technology preventing me from seeing obvious reasons why this would not work..

If each client served the same amount of data they consumed, the web server would only have to serve any given file one single time and the rest of the bandwidth would be picked up by the clients.

Edit: I am aware of P2P and bittorrents. I am asking, why not apply this concept to general web surfing activity?

Answer 1

It is called P2P. The most succesful use is bittorrent. It's very good for big files (Linux distros, open office, etc.). Not so efficient for small files.

World of Warcraft patches are delivered by a similar system aswell.

Answer 2

From the user's perspective, where they get the data is irrelevant so long as it arrives fast, predictably, and securely.

A P2P distribution model is certainly feasible, but it is harder to deliver on those three requirements with P2P compared to client-server.

Answer 3

To "serve" data like this, each client would have to have a server installed on its computer -- and accessible to the world (which is not always easy when you are behind a NAT, a proxy, an entreprise network or whatever).

There are protocols that do that (many Peer to Peer protocols, for instance -- but those work better for big files than for small images, I'd say), but it's just not the way the Web works.

Also, where the data comes from is not what matters to users, as long at is arrives ; and this is your problem, as webmaster/admin.

I should add that with a distributed process like the one you're thinking about, there is a security risk : what if one user decides to send something else than what was really requested ? Say some "not OK" image instead of the real one ? Or a virus ? Or any bad stuff like that ?

Answer 4

The model you describe is the P2P model, and that's exactly how some protocols (for example, BitTorrent, work).

If you're asking why this kind of protocol isn't used for websites or general purpose interned use, there are probably two main reasons:

1. It would probably be very difficult to do, technologically: You'd have to be able to redirect clients to each other in a smart way, you'd have to be able to get clients to actually send files (which would require access to their system, something you generally don't have), you'd probably have to take into accounts things like the speed of each client (i.e. you don't want someone on an old, very slow connection serving up images, as your site would look very slow). There are many more reasons, these are just the most immediate problems I can think of, off the top of my head.
2. It would go over badly with people: Even assuming you could somehow come up with ways of doing this, this kind of thing would probably not go over well with users of your site. Users generally don't want to "share" their computer space/bandwidth/etc. Most times that companies have put "spyware" or other such hidden things on someone's computers, they've gotten very bad publicity.

All in all, I don't think anyone would seriously consider trying to do something like this, mostly for the two reasons I outlined above.

Answer 5

Peer-to-peer networking only works for static content, due to complications with keeping a given file in sync.

Answer 6

P2P is pretty well covered in the other answers but an additional concern is current infrastructure. For consumer class Internet access (cable, DSL, etc) outgoing bandwidth is usually only a fraction of incoming. For example cable download speeds can often be 5-6 Mbps, but upload speeds are often in the 400K range or less.

Internet carriers designed the system this way since most people download content in browsers or other applications and generate a smaller amount of content.

Answer 7

For several reasons:

• Clients tend to be behind NAT, which prohibits serving data directly to other clients.
• Clients tend to have poorer connectivity than servers.
• You can't rely on a given client having suitable available bandwidth, or actually serving what they claim they will. If you care about how your users experience your site, you want to make sure the content is served quickly, not catch-as-catch-can.
• You need a way to verify that the data you're getting hasn't been modified by the client you are getting it from (or the client they got it from, and so on) - this would require digital signatures, which would imply all the same issues and hassles as SSL.
• It would require major reworking of HTTP (if not an outright redesign) to support this. Client support would be spotty at best, especially with little incentive to clients to support it.

Answer 8

You'd have to have something running in the background on someone's computer to serve up whatever content someone else is requesting. How would you get around the spam potential as each client has to say, "Yes, I have that," or "No, I don't have that," which would be some traffic possibly.

Another point not mentioned, is that some sites wouldn't want to let their images out into the public,e.g. porn sites come to mind, so not everyone would want this.

Answer 9

With a little UDP magic, the Adobe Flash Player is now able to lash together with other clients in an ad hoc P2P network. It uses some fairly interesting network chat to punch holes through NAT setups.

http://labs.adobe.com/wiki/index.php/Stratus%3AFAQ

http://en.wikipedia.org/wiki/Real%5FTime%5FMedia%5FFlow%5FProtocol

http://www.adobe.com/devnet/flashplayer/articles/rtmfp%5Fstratus%5Fapp.html

http://www.flashrealtime.com/tuts/p2p-in-flash.html

Answer 10

For most internet applications, the slowest part of the network is the final link to the user's machine. So client P2P has some use - those cases where the server bandwidth/number of users is worse than the last mile bandwidth - but doesn't necessarily work for other cases.

There's an interesting lecture on techniques to avoid duplication of information in routers by Van Jacobson using another level of reference over those in TCP/IP.