tags:

views:

27

answers:

1

Hi!

A small question concerning downloading xml files from a remote server. I am using curl to achieve this.

The information I need in those xml files are located in the first couple of lines of code. Now my question is :

Is it possible to download only a portion ( or a defined amount of bytes ) from these xml files , so I only have the first (let's say) 30 lines.

Those xml files are between 100 kb to 12 mb , so you understand that it would be more efficient to be able to minimize the bandwith of the curl request ( I'm talking about 1000 xml files a day at least ).

Thanks

A: 

Check out the --range option for cURL, which lets you do just that.

For HTTP, this will require the remote server to honour HTTP/1.1 Range requests - for example, if you specify a range -r0-10 then the following header will be transmitted

Range: bytes=0-10

If server doesn't want to honour that, it will return the entire file.

Paul Dixon
thanks! I will check it out.
Paolo Mulder
This may not work if the server you are connecting to doesn't support the CURLOPT_RANGE flag.
GWW
If this is the case , is there any solution to this problem ?
Paolo Mulder
@[Paolo Mulder] its probably best to check the file length after you downloaded it and if it exceeds the expected length because of the server ignoring the range option you truncate it.
Max
@Paulo - yes, if you just want the *start* of a file, a range request isn't necessary. You could simply write a basic HTTP client to send the request and then close the connection when you'd read enough lines. See the PHP manual page on fsockopen, you should find some sample code there.
Paul Dixon