views:

3935

answers:

4

What is the best method using SSIS (SQL Server Integration Services) to upload a file to either a remote SFTP (secure FTP with SSH2 protocal) or FTPS (FTP over SSL) site? I've used the following methods, but each has short-comings I would like to avoid:

COZYROC LIBRARY

Method: Install the CozyRoc library on each development and production server and use the SFTP task to upload the files.

Pros: Easy to use. It looks, smells, and feels like a normal SSIS task. SSIS also recognizes the password as sensitive information and allows you all the normal options for protecting the sensitive information instead of just storing it in clear text in a non-secure manner. Works well with other SSIS tasks such as ForEach Loop Containers. Errors out when uploads and downloads fail. Works well when you don't know the names of the files on the remote FTP site to download or when you won't know the name of the file to upload until run-time.

Cons: Costs money to license in a production environment. Makes you dependent upon the vendor to update their libraries between each version. Although they already have a 2008 version, this caused me a problem during the CTP's of 2008. Requires installing the libraries on each development and production machine.

COMMAND LINE SFTP PROGRAM

Method: Install a free command-line SFTP application such as Putty and execute it either by running a batch file or operating system process task.

Pros: Free, free, and free. You can be sure it is secure if you are using Putty since numerous GUI FTP clients appear to use Putty under the covers. You DEFINATELY know you are using SSH2 and not SSH.

Cons: The two command-line utilities I tried (Putty and Cygwin) required storing the SFTP password in a non-secure location. I haven't found a good way to capture failures or errors when uploading files. The process doesn't look and smell like SSIS. Most of the code is encapsulated in text files instead of SSIS itself. Difficult to use if you don't know the exact name of the file you are uploading or downloading.

A 3RD PARTY C# or VB.NET LIBRARY

Method: Install a SFTP or FTPS library and use a Script Task that references the library to upload the files. (I've never tried this, so I'm going to guess at the pros and cons)

Pros: Probably easy to capture errors. Should work well with variables, so it would probably be easy to use even when you don't know the exact name of the file you are uploading or downloading.

Cons: It's a script task combined with .NET libraries. If you are using SSIS, then you probably are more comfortable with SSIS tasks then .NET code. Script tasks are also difficult to troubleshoot since they don't have the same debugging tools and features as regular .NET projects. Creates a dependency on 3rd party code that may not work between different versions of SQL Server. To be fair, it is probably MORE likely to work between different versions of SQL Server than a 3rd party SSIS task library. Another huge con -- I haven't found a free C# or VB.NET library that does this as of yet. So if anyone knows of one, then please let me know!

+3  A: 

The following question might be of use:

http://stackoverflow.com/questions/152981/recommend-one-favorite-ssis-component-that-does-sftp-ftps

Cozyroc:

It should be easy to test ssh protocol availability by setting the server to "allow only SSHv2" and testing. Have your tried asking Cozy's sales dept?

Command line sftp:

The unknown filename problem could be solved simple scripting/use of wildcards (at least under Cygwin).

3rd party lib:

Why do you need a third party lib for FTPS? .NET has supported this protocol since 2.0 or so.

http://msdn.microsoft.com/en-us/library/system.net.ftpwebrequest.enablessl.aspx

Martin Carpenter
For free solutions, FTPS looks good except when I have to use SFTP. Since I would prefer to have one architecture to handle both scenarios, I'll probably rely on command line sftp. For paid for solutions, CozyRoc, SecureBlackBox, and Rebex all look pretty good. Thanks!
Registered User
+1  A: 

Thank you for the nice review. I want to make 2 comments:

  1. CozyRoc SSH Connection Manager does support SSH2.

  2. There is one pro, which you didn't mention - very, very responsive support and resolution assistance from the CozyRoc team. Many of our clients who have asked questions can confirm my words. Most of the questions get answered in less than 6 hours and we release on-the-spot hot fixes as soon as physically possible.

Thanks for the feedback. We tested out the 1.3 CozyRoc library on Thursday and at this point we'll likely license your product. The CozyRoc team answered some of our questions on Friday and your site also indicated you used SSH2 once we dug into the details further.
Registered User
A: 

I just wanted to provide an update on what we actually decided to do to resolve the SFTP issues in SSIS. Here's the breakdown of what happened:

  1. I initially tried using Putty and some batch files to upload files, but it was difficult to capture errors. Also, I was storing our SFTP credentials in clear text files since it was part of the Putty upload scripts.

  2. We purchased a CozyRoc license for our SSIS server for a couple hundred dollars a year and I'm completely satisified with the results of using their product. With CozyRoc's product, the control flow task raises errors if there are any problems with the upload. Since I also have several junior SSIS programmers on my team, it was easier for them to understand how to setup the control flow task than to use the Putty scripts method. And finally, the password is encrypted using SSIS's native encryption for protecting sensitive data. I no longer have any passwords stored in clear text on my server.

I did review some of the other 3rd party libraries that were recommended in this question, but it appears CozyRoc was the cheapest of the vendors and they also had a few other SSIS tasks that I have been able to use in my BI team. Thanks, CozyRoc!

Registered User
A: 

I recommend this this SSIS SFTP component. They have an article illustrating how to use it in SSIS

Slam Brown