How do you create multiple HTTPDownloader instance with partial download asynchronously? and does it assemble the file automatically after all download is done?
+1
A:
You must use Range HTTP header:
Range. Request only part of an entity. Bytes are numbered from 0. Range: bytes=500-999
Ie. If you want download 1000 file in 4 parts, you will starts 4 downloads:
- 0-2499
- 2500-4999
- 5000-7499
- 7500-9999
And then simply join data from responses.
To check file size you can use HEAD method:
HEAD Asks for the response identical to the one that would correspond to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
Tomasz Wysocki
2010-07-25 10:13:56
I've been looking at the HTTPDownloader class but I only see a gotHeader method. Is it the same thing? Also, can you give me an example code of how to request for the HEAD in twisted?
Marconi
2010-07-26 10:30:44
I'm not twisted programmer, but something like: getPage(url, headers={'range': 'bytes=500-999'})should do the job.Also you can use: getPage(url, method='HEAD')but I don't know what will be result (again I'm not twisted programmer).
Tomasz Wysocki
2010-07-26 11:27:12
I see. Thanks for the info anyway.
Marconi
2010-07-26 14:45:03