How can I throttle a download in a Perl program?

Question

Is there any Perl module available for download throttling? I would like to download a certain file but limit the download rate to a specific number of KB/sec .

Answer 1

A technique not limited to Perl and not limited to a particular protocol is to use trickle:

trickle is a portable lightweight userspace bandwidth shaper. It can run in collaborative mode (together with trickled) or in stand alone mode.

See also http://stackoverflow.com/questions/235762/how-do-you-throttle-the-bandwidth-of-a-socket-connection-in-c

It would be nice to package up this technique as a Perl module (e.g. that subclasses IO::Handle) but I am not aware of one.

Answer 2

Looks like WWW::Curl and the CURLOPT_MAX_RECV_SPEED_LARGE option is what you want:

#!/usr/bin/env perl

use strict;
use warnings;
use feature ':5.10';
use WWW::Curl::Easy;

# Setting the options
my $curl = WWW::Curl::Easy->new;

$curl->setopt(CURLOPT_HEADER,1);
$curl->setopt(CURLOPT_URL, 'http://www.google.com');
$curl->setopt(CURLOPT_MAX_RECV_SPEED_LARGE, 1);

my $response_body;
open my $fh, ">", \$response_body or die; # presumably this can be a real file as well.
$curl->setopt(CURLOPT_WRITEDATA,$fh);

my $ret = $curl->perform;
die 'Error: '. $curl->strerror($ret) if $ret;

my $response_code = $curl->getinfo(CURLINFO_HTTP_CODE);
say "Received response: $response_body";

In this example, we download Google at one byte per second. Very slow.

Answer 3

If a recent version of wget is installed on your system, this command has a --limit-rate option to download from http, https, and ftp URLs.