FTP Timout on NLST when directory is empty

Question

EDIT: Learned that Webmethods actually uses NLST, not LIST, if that matters

Our business uses the WebMethods integration server to handle most of our outbound communications, and its FTP functionality leaves something to be desired. We are having a problem that may be specific to WebMethods, but if anyone can point me in a direction of what kinds of things might cause this I'd appreciate it.

When polling two of our partners' FTP servers, we connect without issue but, when doing a NLST on a directory that is empty (no files and no subdirectories) it's timing out. The actual error is:

com.wm.net.ftpCException: [ISC.0064.9010] java.net.SocketTimeoutException: Accept timed out

It's being thrown during the invocation of the pub.client.ftp:ls service. I've logged in with a number of FTP clients without a problem to the same sites. I've used whatever the default FTP client is in windows, FileZilla and lftp. All without issue. The servers themselves aren't the same FTP server software from what I can tell. One is Microsoft FTP, the other I'm uncertain on but is definitely not Microsoft.

Any idea what could cause an FTP client to timeout when waiting for a NLST response on an empty directory? The visible responses from the FTP server appear to be the same, but is there a difference in how NLST responds for an empty directory that I'm unaware of?

This problem is consistent on these two servers. Everything functions fine on directories with files or subdirectories within it, but not when empty.

Any thoughts or directions would be appreciated.

Thanks!

Eric Sipple

Answer 1

I am not sure if it was the same problem but I had similar symptoms a while ago using another FTP client in Java (commons.net). The problem turned out to be caused by the active/passive mode of the connection. I am sorry I can't give you more details, that's all I can remember... hope that help.

Answer 2

Guillermo Vasconcelos was correct in his answer. There are two FTP modes, Active and Passive. The default FTP mode is active. Active requires the server to connect back to the client on some TCP/IP port. This does not work with firewalls because chances are that this port would be blocked or if you are behind a Router with NAT, not mapped.

If you use Passive (PASV) mode instead, you should not get the hang.

Answer 3

I'm going to run some new tests with the settings to passive tomorrow when maintenance is done here, but I'm not sure that's the issue. We are able to get a directory listing if there are files or subdirectories in that directory. It only fails when the directory we're NLST-ing on is empty.

Would the active/passive difference only manifest for an empty directory, or is there another possibility?

Answer 4

I tried this in WebMethods IS Version 6.5 Updates WmPRT_6-5-1_SP1, IS_6-5_SP3.

It worked perfectly first time.

I turned on debugging on the FTP server (Debian's default ftpd). WebMethods' NLST honours the active/passive parameter passed to it.

There's nothing special about the NLST command, nor its correct behaviour with an empty directory -- if LIST works, then so should RETR, STOR and NLST. If NLST works with a non-empty directory, it should work with an empty one.

So my guess is that either:

• Your version of WM has a bug mine doesn't
• Your FTP server has a bug mine doesn't
• There's a wacky protocol-aware firewall in your system that doesn't like FTP data sockets with no data in them.

Firewall vendors are a bit wayward when it comes to FTP... When testing with other clients, make sure it's from the same machine on which WebMethods Integration Server is running.

Just for the record, here's what should happen for an active NLST

• client opens a listening socket, and sends a PORT command with that socket's details
• client sends NLST command
• server connects to client's listening socket (this is the data socket)
• server transmits listing over data socket (in this case, zero bytes)
• server closes data socket

... and in passive mode

• client sends PASV command
• server opens a listening socket, and replies with PASV response containing its details
• client connects to listening socket (this is the data socket)
• client sends NLST command
• server transmits listing over data socket (zero bytes again)
• server closes data socket

Answer 5

FTP requires that both the specified port and the one above it be opened through the firewall. When I had problems with webMethods timing out, it was because the firewall did not have the return port open.

Howard