Upload of very large files

Question

Hi there,

I am looking for a way to upload very large files, means, 5 GB or more, via a Web-Interface. The language/server system can be chosen. POST-Requests seem not to work since the browser-internal filepointer only handles files up to 2 GB. Other options would be e.g. an Java-applet, a FTP-Request (for example done by a Javascript-Library) or anything else that can do it reliably.

Any ideas appreciated.

Answer 1

Anything that runs inside the browser won't work because it will use the same file API. So your options are a standalone app or an applet or maybe Flash (which I don't know).

I suggest to try an applet and commons-fileupload.

Answer 2

Just a thought ... HTTP is not apppropriate for data transfers of that size.

Although it may work, it's most likely to fail and would be difficult to resume.

FTP sounds more promising, but gigabyte uploads are pushing it.

Bittorrent handles this size of transfer, but that's a whole 'nother story.

Answer 3

I'd same task here; we ended with a Silverlight client application which performs following activities:

• Allows user select one or more files
• For each file
  • Break it in small data chunks (just few kbytes)
  • Compress chunk data with gzip
  • Create a checksum and send it to a webservice
  • At server-side, check that checksum
  • Create/Open partial file
  • Append uncompressed data to end
  • Report progress with a progress bar and firing Javascript events

So far, works like a charm.