views:

572

answers:

6

Hello,

I'm trying to download files in subdirectories from an ftp server with ant. The exact set of files is known. Some of them are in subdirectories. Ant only seems to download the ones in the root directory. It does work if I download all files without listing them.

The first ftp action should do the exact same thing as the second. Instead I get "Hidden file \\a\a.txt assumed to not be a symlink."

Does anyone know what's wrong here? Is this a bug in the ant FTP task?

<?xml version="1.0" encoding="utf-8"?>
<project name="example" default="example" basedir=".">
    <taskdef name="ftp" 
    classname="org.apache.tools.ant.taskdefs.optional.net.FTP" />

    <target name="example">

        <!-- doesn't work -->
        <ftp action="get" verbose="true"
        server="localhost" userid="example" password="example" 
        remotedir="">
            <fileset dir="downloads" casesensitive="false" 
            includes="a/a.txt,a/b/ab.txt,c/c.txt" />
        </ftp>

        <!-- works (but requires multiple ftp tasks) -->
        <ftp action="get" verbose="true"
        server="localhost" userid="example" password="example"
        remotedir="a">
            <fileset dir="downloads" casesensitive="false" 
            includes="a.txt,b/ab.txt" />
        </ftp>
        <ftp action="get" verbose="true"
        server="localhost" userid="example" password="example"
        remotedir="c">
            <fileset dir="downloads" casesensitive="false" 
            includes="c.txt" />
        </ftp>

    </target>

</project>

Update: I posted a bug about this to Commons Net jira https://issues.apache.org/jira/browse/NET-324

Update: I added a bugreport to the ant bugreport system https://issues.apache.org/bugzilla/show_bug.cgi?id=49296

A: 

Did you already try to set '/' as value of remotedir?

<ftp action="get" verbose="true"
   server="localhost" userid="example" password="example" 
   remotedir="/">
  <fileset dir="downloads" casesensitive="false" 
    includes="a/a.txt" />
</ftp>
CodeSeavers
Yes I did. That doesn't solve it.
Kristof Neirynck
A: 

I don't know that setting remotedir="" is legitimate. If you want to use the default root directory then you should leave it out entirely.

Zac Thompson
I tried that too. It doesn't work.
Kristof Neirynck
A: 

As I need this to work, I've made this workaround. It seems to work, but I'm not entirely pleased with it. It doesn't feel clean.

<scriptdef name="my-ftp-get" language="javascript">
    <attribute name="server"/>
    <attribute name="userid"/>
    <attribute name="password"/>
    <attribute name="remotedir"/>
    <attribute name="fileset_dir"/>
    <attribute name="fileset_includes"/>
    <![CDATA[
    importClass(java.io.File);
    importClass(org.apache.tools.ant.taskdefs.optional.net.FTP);
    var local_basedir = "" + attributes.get("fileset_dir") + "/";
    var original_includes = "" + attributes.get("fileset_includes");
    var remotedir = "" + attributes.get("remotedir");
    local_basedir = local_basedir.replace(/\\/g, "/");
    original_includes = original_includes.replace(/\\/g, "/");
    remotedir = remotedir.replace(/\\/g, "/");
    var includes_arr = original_includes.split(",");
    var clean_includes = {};
    for (var i = 0; i < includes_arr.length; i++) {
        var directory = "/";
        var filename = includes_arr[i];
        var split_include = includes_arr[i].split("/");
        if (split_include.length > 1) {
            directory = split_include[0] + "/";
            filename = includes_arr[i].substring(directory.length);
        }
        if (!clean_includes.hasOwnProperty(directory)) {
            clean_includes[directory] = [];
        }
        clean_includes[directory].push(filename);
    }
    var get_files = new FTP.Action();
    get_files.setValue("get");
    for (var path in clean_includes) {
        var current_clean_includes = clean_includes[path].join(",");
        var fileset = project.createDataType("fileset");
        var ftp = self.project.createTask("ftp");
        ftp.setAction(get_files);
        ftp.setServer(attributes.get("server"));
        ftp.setUserid(attributes.get("userid"));
        ftp.setPassword(attributes.get("password"));
        ftp.setRemotedir(remotedir + path);
        fileset.setDir(new File(local_basedir + path));
        fileset.setIncludes(current_clean_includes);
        ftp.addFileset(fileset);
        ftp.perform();
    }
    ]]>
</scriptdef>

<my-ftp-get
server="localhost" userid="example" password="example"
remotedir=""
    fileset_dir="downloads" casesensitive="false" 
    fileset_includes="a/a.txt,a/b/ab.txt">
</my-ftp-get>
Kristof Neirynck
A: 

The first case will try to get the file //downloads/a/a.txt The second case will try to get the file //a/downloads/a.txt

If you replace the one that doesn't work with

<ftp action="get" verbose="true"
    server="localhost" userid="example" password="example" 
    remotedir="">
        <fileset dir="a/downloads" casesensitive="false" 
        includes="a.txt" />
</ftp>

or

<ftp action="get" verbose="true"
    server="localhost" userid="example" password="example" 
    remotedir="">
        <fileset dir="a" casesensitive="false" 
        includes="downloads/a.txt" />
</ftp>

it should work better. If these do not suit your needs could you provide the full path of the file(s) you want to match and maybe we can create some alternatives.

MatsT
No, you are confounding the remoteDir and the fileset dir attribute. To quote the manual: "Getting files from an FTP server works pretty much the same way as sending them does. The only difference is that the nested filesets use the remotedir attribute as the base directory for the files on the FTP server, and the dir attribute as the local directory to put the files into."
Kristof Neirynck
I want to download a few files in the root directory and several files in different subdirectories. To give an example: root.txt, a/a.txt, a/b/ab.txt, c/c.txt
Kristof Neirynck
A: 

You may have more success with the Apache Commons VFS Ant Tasks. These provide reading/writing to various file systems, with copying. synchronization and other operations. This will also be a flexible solution should you need to change to use sftp, ssh, http or some other system later; unlike the native ant tasks for ftp, ssh etc, the majority of the VFS configuration is independent of filesystem, with filesystem specifics mostly requiring just a change of URL.

Your example would be coded along these lines:

<taskdef resource="org/apache/commons/vfs/tasks/tasks.properties"/>

<target name="example">
     <v-copy destdir="${basedir}" srcdir="ftp://example:example@localhost/downloads" includes="a/a.txt,a/b/ab.txt,c/c.txt"/>
</target>
mdma
A: 

I believe to have found and fixed the problem. Also see my the bugreport in jira: https://issues.apache.org/jira/browse/NET-324

and I have added a bugreport to the ant bugreport system https://issues.apache.org/bugzilla/show_bug.cgi?id=49296

diff --git a/libs/apache-ant-1.8.1/src/main/org/apache/tools/ant/taskdefs/optional/net/FTP.java b/libs/apache-ant-1.8.1/src/main/org/apache/tools/ant/taskdefs/optional/net/FTP.java
--- a/libs/apache-ant-1.8.1/src/main/org/apache/tools/ant/taskdefs/optional/net/FTP.java
+++ b/libs/apache-ant-1.8.1/src/main/org/apache/tools/ant/taskdefs/optional/net/FTP.java
@@ -834,8 +834,7 @@
                         } else if (!result) {
                             return;
                         }
-                        this.curpwd = this.curpwd + remoteFileSep
-                            + currentPathElement;
+                        this.curpwd = getCurpwdPlusFileSep() + currentPathElement;
                     } catch (IOException ioe) {
                         throw new BuildException("could not change working dir to "
                                                  + (String) pathElements.elementAt(fcount)
@@ -895,7 +894,7 @@
              * @return absolute path as string
              */
             public String getAbsolutePath() {
-                return curpwd + remoteFileSep + ftpFile.getName();
+                return getCurpwdPlusFileSep() + ftpFile.getName();
             }
             /**
              * find out the relative path assuming that the path used to construct
@@ -1036,6 +1035,17 @@
             public String getCurpwd() {
                 return curpwd;
             }
+
+            /**
+             * @return parent directory of the AntFTPFile with a remoteFileSep appended
+             */
+            private String getCurpwdPlusFileSep() {
+                if (this.curpwd.endsWith(remoteFileSep)) {
+                    return this.curpwd;
+                }
+                return this.curpwd + remoteFileSep;
+            }
+            
             /**
              * find out if a symbolic link is encountered in the relative path of this file
              * from rootPath.
Kristof Neirynck