views:

132

answers:

3

Lately I've asked this question. But the answer doesn't suit my demands, and I know that file hosting providers do manage to limit the speed. So I'm wondering what's the general algorithm/method to do that (I do mean downloading technique) - in particular limiting single connection/user download speed.

@back2dos I want to give a particular user a particular download speed (corresponding to hardware capabilities of course) or in other words give user ability to download some particular file with lets say 20kb/s. Surely I want to have an ability to change that to some other value.

+3  A: 

You could use a token bucket ( http://en.wikipedia.org/wiki/Token_bucket)

Joshua Smith
Or a leaky bucket...
WhirlWind
Yes, though the token bucket allows for bursts. You can also use a token buck in a hierarchical setup easily and cleanly (you could probably use a leaky bucket in a hierarchy,too, but the hard limit of the leaky bucket, IMHO, complicates that).
Joshua Smith
+2  A: 

Without mention of platform/language, it's difficult to answer, but a "leaky bucket" algorithm would probably be the best fit:

http://en.wikipedia.org/wiki/Leaky_bucket

spender
A: 

Well, since this answer is really general, here's a very simple approach for plain TCP:

You put the resource handlers of all download connection into a list, paired up with information about what data is requested, and loop through it. Then you write a chunk of the required data onto the socket, maybe about 1.5K, which is the most commonly used maximum segment size, as far as I know. When you're at the and of the list, you start over. Before starting over, simply wait to get the desired average bandwidth.

Please note, if too many clients have lower bandwidth than you allow, then your TCP buffer is likely explode. some TCP bindings permit finding the size of currently buffered data for one socket. if it exceeds a threshold, you can simply skip the socket.

Also, if too many clients are connected, you will actually not have enough time to write to all the sockets, thus after one loop, you "have to wait for a negative time". Increasing the chunk size might speed up things in such scenarios, but at some point your server will stop getting faster.

A more simple approach is to do this on the client side, but this may generate a lot of overhead. The dead simple idea is to have the client request 1K every 50ms (assuming you want 20KB/s). You can even do that over HTTP, although I strongly suggest bigger chunk size, since HTTP has enourmous overheads.

My guess is, the best is to try to find a webserver capable of doing such things out of the box. I think Apache has a number of modules for al kinds of quota.

greetz
back2dos

back2dos