PHP uploader for about a thousand images?

Question

I am working on a websitwe and I need a client to send me close to a thousand pictures. There is a bit of a geographical distance between us, so while mailing a CD or SD card is not impossible, it is not preffered. Can I write a PHP uploader to have the client submit the pictures, or is there a better way? I do not want to subscribe to any file sharing sites.

Assuming I am writing such an uploader, is there a way to code a progresbar and/or some other features that protect against corrupted files by interrupted upload?

Any links to resources that help are a plus.

Thanks

EDIT:

This is a one time thing and these are high quality images we are discussing. It's gonna be a few GB.

By FTP Space, you mean a username and pwd to my FTP server? It can be done, but (why) is an FTP client better than a web based uploader? I am not sure that the client knows how to or wants to download an FTP program. The client is on a Mac and I don't trust the few FTP programs for Mac that I've seen.

Assuming I take the "everything in a single zip" route, will PHP work?

Thanks again.

EDIT:

Okay, so far I've been convinced not to use PHP unless we use .zip files. Also, I probably will nix the progress bar for now. Waiting for client, will get back to everyone in the next few days.

EDIT:

Met the client at some sort of function. The client had the photos there on a flash drive. Problem solved. +1 for all of the input.

Answer 1

Disregarding the number of files, how much data are we talking about?

If you're concerned about the number of files, just have the client zip them. You can decompress the zip on the server and put the images wherever they need to go.

Also, ask yourself if this is a process that is going to happen regularly, or is it an irregular maintenance or one-time startup task? If so, you'll save yourself and your client time and money by just exchanging the files via ftp or some other method.

Answer 2

I would say set up some FTP space, or a WebDAV share. If you can't do that then I would upload the images in a single (or small number) of zip files, rather than trying to deal with hundreds of unique uploads, which would be very time consuming as browsers will only select a single file at a time in a file upload dialog (i.e. you can't just select an entire directory to upload).

Why not to use PHP in this instance:

Writing a simple file upload handler is easy in PHP but if you do want to handle very large uploads it's quite a bit tricker as you will need to be wary of file size upload limits, script excution time limits and memory limits. It can be a bit frustrating to setup the first time.

If it's a multi GB file you want to handle then an HTTP upload is not the way you want to go (based on, that if you have to ask why, getting it working as you want is probably going to be very frustrating process for you).

There are several excellent FTP clients on Mac OS, and there is no reason to trust a web browser to be better method of transferring files than an FTP Client.

Regarding progress bar support:

Currenly progress bar support is very limited in PHP (currently only via 3rd party modules which are a little hairy) though I belive there is a plan to include the module in the standard distribution. Googling for uploadprogress.so should give some relevant results.

In reply to littlegreen:

I used this extension http://pecl.php.net/package/uploadprogress (which managed to conflict with at least one extension I had installed already - although I think they have fixed that issue now) and wrote a custom javascript handler (http://iaincollins.com/javascript/FileUploader.js) based on some multiple examples on the web, in an effort to incorporate the best of each. e.g. Using an invisible iframe to return the upload progress transparently, creating a DHTML file upload dialog box and replacing the ugly browser based file upload control by making it transparent and overlaying it on top of my own control (so it was still clickable, as you can't invoke the file upload dialog from JavaScript, although you can get the name of the file that has been selected for upload). Getting the CSS styling right was particularly important for this - it's quite browser specific.

The JS I wrote is sadly very situation specific, but it might have some helpful elements - sorry I don't have anything public I can link to as it was created for a private interface.

I see there is a jQuery plugin for uploadprogress now - it doesn't do everything, but it would be a good start: http://nixbox.com/demos/jquery-uploadprogress.php ... I recall better examples being out there, including one that handled multiple file uploads (by queuing them and performing them sequentially - you still had to select one file at a time) but unfortunately I can't seem to find them.

Conclusion:

I would try and avoid the hassle and do something like FTP space or a private / peer to peer file transfer application unless you have the time.

File Transfer Protocol (as the name implies) is specifically designed for large file transfer and is a better option as it will need little configuration (you won't run into resource limits) and you can reasonably expect the client to have FTP resume support, assuming your server does (i.e. if the download stalls, they should be easily able to resume it from where they left off, rather than starting from scratch again as they would have to do with a PHP file based HTTP upload).

While HTTP has the ability to handle uploading files it's not as robust, and various resource limits (in the web server, and in PHP - as mentioned above) would need to be explicitly configured to make the uploading of a multi GB file possible. Even with WebDAV (using HTTP sever a remote file system) it's transparent but it tends not to be as robust as FTP, chiefly due to buggy client implementations - specifically there can be problems when transferring very large files on both Windows and Mac OS, although it's typically fine for small (e.g. <500 MB) file transfers.

As a side note, the HTML5 specification will improve how file uploads are handled - with the possibility to upload multiple files at once, and with file upload progress as part of the specification, but for now it's not really an option.

Answer 3

I would suggest FTP, with the reason being that you can do bulk uploads, start/pause/stop and preserve folder hierarchy (if necessary). The FTP solutions for Mac are great; I use Transmission and Cyberduck. Filezilla is also good.

Another FTP option would be for your client to use his Mac to set up a local FTP server, from which you can download the photos. This would require a bit of configuration, but is definitely possible.

In any case, I wouldn't recommend a browser-based file upload for this amount of files. If you upload them one-by-one (or even in batches), you're likely to create a whole lot of hassle for your client. PHP's post parameters and upload file size restrictions will make compressing all images into one archive difficult, and transferring large files can cause timeout issues as browsers aren't generally capable of handling pausing and resuming.

The exception to browser-based uploads is to use a Flash-based file uploader, such as SWFUpload. With it, you can have your client select all files in one go, after which the files will be queued and uploaded one-by-one to your server-side script (which can be in PHP). In fact, I've used this for the very same purpose of letting client send me files.

Having said that, I still recommend FTP in this instance.

Answer 4

I just had to solve a similar problem for a client. They regularly have THEIR clients upload a bunch of files to them, totalling 300-400mb, normally through FTP. However, one client had a firewall and could not use FTP. And so I had to code a HTTP uploader.

I searched around the Internet for a PHP solution and found basically the same things as Ian Collins already said, 1) PHP is possible but hairy and 2) Progress bar support for PHP is bad. I can give you a bit more details though, about upload/size limits you should be fine as long as you include a .htaccess file in your web directory containing the following code, which should give you an upload limit of 3GB and unlimited script execution time as long as your webhost supports setting PHP conf values in htaccess files:

php_value upload_max_filesize 3000M
php_value post_max_size 3000M
php_value max_execution_time 0
php_value max_input_time 0

Coding an uploader script is relatively simple. A good tutorial is found here.

Using a progress bar is more tricky. A PHP-only progress bar is not possible without installing extensions. A generally accepted option is to use a hybrid Perl/PHP solution, which however requires you to be able to run CGI scripts on your server. I can't, so I've left the progress bar out. This is highly client unfriendly though, because he's staring at a non-moving screen for quite some time, thinking the browser has locked up, and if he somehow closes it or loses his Internet connection, he'll have to start over again.

Uploading several files at once is also tricky. By using an advanced file selection control and AJAX to open several concurrent uploading session you could come a long way, but you would be left with the same problem: when it fails because of some weird filename, disk full or connection lost, the client will have to start over again but there will be a mess of files already on the server. Coding a userfriendly resume solution is not an easy task. I'd advise you to let the client ZIP his files and upload them in a bulk.

Since this is a one-time thing, I would advise you not to bother with writing your own solution and use tools already available. Why don't you let your client use FTP, for instance the MAC version of FileZilla? Or DropBox? Or how about letting him upload the files to Google Docs, which since recently allows uploading and sharing any file up to 100mb?