Java File Transfer API

Question

I need to transfer files to my web server for processing and I'd like to do it in a generic way if possible.

I need to be able to transfer files from the following protocols at a minimum (with more to follow eventually):

HTTP
FTP
SCP

I'd really like to be able to send files to SMTP also

So my question, is there a toolkit available that does this already? If so, it must be open source as this is part of an open source project.

If there isn't a toolkit that already does this, what is the best way to structure an interface that will handle most file transfers?

I've thought about something like this:

public interface FileTransfer {
    public void connect(URL url, String userid, String password);
    public void disconnect();
    public void getFile(String sourceFile, File destFile);
    public void putFile(File sourceFile, File destFile);
}

And then a Factory that takes the source URL or protocol and instantiates the correct file handler.

Answer 1

Would this help? http://commons.apache.org/net/

Answer 2

Apache commons VFS speaks to this problem, although a quick check didn't show that it will do SCP or SMTP. Commons NET does SMTP, but I don't know that you could get the common interface out of the box. For SCP, here are some possibilities.

The bottom line seems to be to check out the VFS implementation and see if it does something for you, perhaps you can extend it for different protocols. If it isn't appropriate, regarding your interface, you are probably going to want all remote file references to be Strings rather than File objects, and specifically a string representing a URI pointing to the remote location and telling you what protocol to use.

Answer 3

I'm working at a problem very similar to yours, I couldn't find any open source solution so I'm trying to sketch a solution myself. This is what I've come up with.

I think you should represent inputSources and outputSources as different things, like

public interface Input{
      abstract InputStream getFileInputStream();
      abstract String getStreamId();
}
//You can have differen implementation of this interface (1 for ftp, 1 for local files, 1 for Blob on db etc)  

public interface Output{
      abstract OutputStream getOutputStream();
      abstract String getStreamId();
}
//You can have differen implementation of this interface (1 for ftp, 1 for local files, 1 for mailing the file etc)

Then you should have a Movement to describe which input should go to which output.

 class Movement{
        String inputId;
        String outputId;
 }

A class to describe the list of Movement to make.

class MovementDescriptor{
       public addMovement(Movement a);
       public Movement[] getAllMovements();
}

And then a class to perform the work itself.

class FileMover{

      HashMap<String,Input> inputRegistry;
      HashMap<String,Output> outputRegistry;

      addInputToRegistry(Input a ){
          inputRegistry.put(a.getId(),a);
      }
     addOutputToRegistry(Output a){
          outputRegistry.put(a.getId(),a);
      }

     transferFiles(MovementDescriptor movementDescriptor){

         Movement[] movements =movementDescriptor.getAllMovements();
         foreach (Movement movement: movements){
              //get the input Id
              //find it in the registry and retrieve the associated InputStream
              //get the output Id
              //find it in the registry and retrieve the associated OutputStream
              //copy the stream from the input to the output (you may want to use a temporary file in between)
          }
     }
}

The code that would use this would operate like this:

FileMover fm=new FileMover();

//Register your sources and your destinations
fm.addInputToRegistry(input);
fm.addOutputToRegistry(output)

// each time you have to make a movement create a MovementDescriptor and call
fm.transferFiles(movementDescriptor)

If you would like to exchange by mail our views on the subject, just send me an e mail at (my nickname)@gmail dot com.

NOTE: The code is just a sketch :-)

Answer 4

I think JSch implements SCP, so that covers that one.