Read header data from files on remote server

Question

Hi!

I'm working on a project right now where I need to read header data from files on remote servers. I'm talking about many and large files so I cant read whole files, but just the header data I need.

The only solution I have is to mount the remote server with fuse and then read the header from the files as if they where on my local computer. I've tried it and it works. But it has some drawbacks. Specially with FTP:

• Really slow (FTP is compared to SSH with curlftpfs). From same server, with SSH 90 files was read in 18 seconds. And with FTP 10 files in 39 seconds.
• Not dependable. Sometimes the mountpoint will not be unmounted.
• If the server is active and a passive mounting is done. That mountpoint and the parent folder gets locked in about 3 minutes.
• Does timeout, even when there's data transfer going (guess this is the FTP-protocol and not curlftpfs).

Fuse is a solution, but I don't like it very much because I don't feel that I can trust it. So my question is basically if there's any other solutions to the problem. Language is preferably Ruby, but any other will work if Ruby does not support the solution.

Thanks!

Answer 1

What type of information are you looking for?

You could try using ruby's open-uri module. The following example is from http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/index.html

require 'open-uri'
open("http://www.ruby-lang.org/en") {|f|
  p f.base_uri         # <URI::HTTP:0x40e6ef2 URL:http://www.ruby-lang.org/en/&gt;
  p f.content_type     # "text/html"
  p f.charset          # "iso-8859-1"
  p f.content_encoding # []
  p f.last_modified    # Thu Dec 05 02:45:02 UTC 2002
}

EDIT: It seems that the op wanted to retrieve ID3 tag information from the remote files. This is more complex.

From wiki: This appears to be a difficult problem.

On wiki:

Tag location within file

Only with the ID3v2.4 standard has it been possible to place the tag data at the end of the file, in common with ID3v1. ID3v2.2 and 2.3 require that the tag data precede the file. Whilst for streaming data this is absolutely required, for static data it means that the entire audio file must be updated to insert data at the front of the file. For initial tagging this incurs a large penalty as every file must be re-written. Tag writers are encouraged to introduce padding after the tag data in order to allow for edits to the tag data without requiring the entire audio file to be re-written, but these are not standard and the tag requirements may vary greatly, especially if APIC (associated pictures) are also embedded.

This means that depending on the ID3 tag version of the file, you may have to read different parts of the file.

Here's an article that outlines the basics of reading ID3 tag using ruby for ID3tagv1.1 but should server as a good starting point: http://rubyquiz.com/quiz136.html

You could also look into using a ID3 parsing library, such as id3.rb or id3lib-ruby; however, I'm not sure if either supports the ability to parse a remote file (Most likely could through some modifications).