Reading chunks of a remote file without transferring the whole file ?

Question

Is it possible via any Unix utility to read chunks of a remote file (like say 'head' does) instead of transferring the whole file ?

Answer 1

You can execute remote commands through SSH

ssh [email protected] "head <filename>"

Answer 2

1. You can remotely extract interesting part of the file (using dd for example) and then transfer the interesting chunk back.

2. Other option would be to employ netcat:

   dd if=file skip=... bs=... count=...| nc <host> <port>
   

   And on the receiver:

   nc -l -p <port> | dd of=chunk
   

3. You can use curl with the following options (assuming you have got HTTP or FTP that supports seeking):

   • -r to retrieve the range of bytes
   • -C to start downloading from a given offset

For sure there are other possibilities apart from the ones mentioned. More information on your case could help to devise a smarter method.

Answer 3

If you understand Python, the paramiko library should make this reasonably easy to do. It is a pure Python implementation of the ssh protocol, and specifically sftp should allow you to get the part of a file you need.

See http://www.lag.net/paramiko/ for details.