Strange behavior when running a bash script from java program

Question

I have a script (KSH script to be precise) that downloads 3 files from a FTP server, using the curl command.

When I run manually my script, i.e. by executing the command ./ftp_download.sh XXX (XXX are the parameters for the script), the download finishes correctly.

As I want to run the script from a Java program, I have made a short Java class that contains exactly that:

public class Run {

    private static final String CMD = "/.../sh/ftp_download.sh XXX";

    public static void main(String[] args) {
        System.out.println("========================================================");
        BufferedReader out = null;
        try {
            long startTime = System.currentTimeMillis();
            String strOutputline;
            SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
            Date now = new Date();
            Process processus = Runtime.getRuntime().exec(CMD);
            out = new BufferedReader(new InputStreamReader(processus.getInputStream()));
            while ((strOutputline = out.readLine()) != null) {
                now.setTime(System.currentTimeMillis());
                System.out.println(sdf.format(now) + " " + strOutputline);
            }
            System.out.println("RESULT : " + processus.waitFor());
            out.close();
            processus.destroy();
            long duration = System.currentTimeMillis() - startTime;
            System.out.println("Duration : " + duration);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (out != null) {
                    out.close();
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        System.out.println("========================================================");
        System.out.println("END");
    }

}

However, when I run this simple program, it simply freezes after exactly 3m20 (this duration is always the same, even if I run the Java program several times). By freezing, I mean that the Java program is still running (the curl process too) but the downloaded file is not growing anymore (i.e. curl does not continue to download any data)...

Thus, I never get the RESULT: xxx line printed in the console...

What can explain this strange behavior?

ps: In the near future, I will change my project in order to download these files using the Apache commons-net library, but I really want to understand this strange behavior!

---

Thanks to derobert, I finally manage to solve this issue. Some explanations: in normal mode, curl is displaying a progress information (a table with the amount of data downloaded, the remaining time, etc.). After some time, the buffer seems to be completly filled, and that's why the process freezes...

Answer 1

I'm not a Java person, but rather a Unix one, and one thing seems obvious: The buffer on either stdout or stderr is filling up, and then curl is blocking.

Does it work if you run curl in silent mode, as in curl --silent?

Checking the Java documentation, it looks like you want to also use getErrorStream in addition to getInputStream.

Answer 2

From Process:

The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.

So you need to get the standard output and the error output of the process and read them until they’re empty (i.e. returning -1 on read()).