Download files one at a time, not all at once

Question

What I'm trying to do is download a set of images, about 200 of them. Now, I could install some addon or a script to firefox that just lets me download em all with a few clicks, but before I thought of doing that, I was thinking on how I could do this with a few lines of code.

Languages I have at my disposal are C#, Java and PHP. I'd prefer to do this in PHP, since I've never worked with curl before.

The images are neatly ordered, I've already checked that. The filename is a string that remains the same over the entire series, and appended at the end there is .1(point 1), .2, .3, and so on...

So I was thinking of simply using a foreach loop, but I'm not certain as to how this will execute. Even having never worked with curl before, something tells me that if I don't tell curl to wait for each image to finish, it's going to just rush through the foreach loop and as a result, try to download them all at once.

How does this work when using curl? Do I need to set something, or are my worries unjust and has my study of the documentation proven wrong?

And while I'm at it: anything else I should know, using curl for the first time?

Answer 1

curl exits once the download is done. So if you call curl using PHP's system call (which waits until the subprocess you're calling has finished), the images will be downloaded one after the other.

Answer 2

Hi,

I would sugest you to go with PHP and curl. Here is a nice class which you can use: http://www.bitrepository.com/download-image.html

The class by default is setup to wait 60 second to allow the image to properly download.

All you need is to do more is to include a loop for names of the images.

Answer 3

Don't worry, libcurl is synchronously. Even if you were to use libcurl asynchronously (or any other library for that matter), you could just download the next image at the end of the previous image's callback.

In PHP you could just loop file_get_contents I presume. C# and Java also have non-curl network interfaces.

Answer 4

In C# you can use the WebClient class for this kind of work, in particular its DownloadFile method, that is synchronous; by the way, the WebClient provides also asynchronous methods.
However, on Linux I'd use bash and wget.