Downloading files from remote server

Question

Hello,

I am using C# and a console app and I am using this script to download files from a remote server. There area a couple of things I want to add. First, when it writes to a file, it doesn't take into consideration a newline. This seems to run a certain amount of bytes and then goes to a newline. I would like it to keep the same format as the file it is reading from. Second, there are multiple .jpg files on the server that I need to download. How can I use this script to download multiple, .jpg files

public static int DownLoadFiles(String remoteUrl, String localFile)
    {
        int bytesProcessed = 0;

        // Assign values to these objects here so that they can
        // be referenced in the finally block
        StreamReader remoteStream = null;
        StreamWriter localStream = null;
        WebResponse response = null;

        // Use a try/catch/finally block as both the WebRequest and Stream
        // classes throw exceptions upon error
        try
        {
            // Create a request for the specified remote file name
            WebRequest request = WebRequest.Create(remoteUrl);
            request.PreAuthenticate = true;
            NetworkCredential credentials = new NetworkCredential("id", "pass");
            request.Credentials = credentials;

            if (request != null)
            {
                // Send the request to the server and retrieve the
                // WebResponse object
                response = request.GetResponse();
                if (response != null)
                {
                    // Once the WebResponse object has been retrieved,
                    // get the stream object associated with the response's data
                    remoteStream = new StreamReader(response.GetResponseStream());

                    // Create the local file
                    localStream = new StreamWriter(File.Create(localFile));

                    // Allocate a 1k buffer
                    char[] buffer = new char[1024];
                    int bytesRead;

                    // Simple do/while loop to read from stream until
                    // no bytes are returned
                    do
                    {
                        // Read data (up to 1k) from the stream
                        bytesRead = remoteStream.Read(buffer, 0, buffer.Length);

                        // Write the data to the local file
                        localStream.WriteLine(buffer, 0, bytesRead);

                        // Increment total bytes processed
                        bytesProcessed += bytesRead;
                    } while (bytesRead > 0);
                }
            }
        }
        catch (Exception e)
        {
            Console.WriteLine(e.Message);
        }
        finally
        {
            // Close the response and streams objects here
            // to make sure they're closed even if an exception
            // is thrown at some point
            if (response != null) response.Close();
            if (remoteStream != null) remoteStream.Close();
            if (localStream != null) localStream.Close();
        }

        // Return total bytes processed to caller.
        return bytesProcessed;

Answer 1

The reason you're getting spurious newlines in the result file is because StreamWriter.WriteLine() puts them there. Try using StreamWriter.Write() instead.

Regarding downloading multiple files, can't you just run the function several times, passing it the URLs of the different files you need?

Answer 2

Why don't you use WebClient.DownloadData or WebClient.DownloadFile instead?

WebClient client = new WebClient();
client.Credentials = new NetworkCredentials("id", "pass");
client.DownloadFile(remoteUrl, localFile);

By the way the correct way to copy a stream to another is not what you did. You shouldn't read into char[] at all, as you might run into encoding and end of line issues as you are downloading a binary file. The WriteLine method call is problematic too. The right way to copy contents of a stream to another is:

void CopyStream(Stream destination, Stream source) {
   int count;
   byte[] buffer = new byte[BUFFER_SIZE];
   while( (count = source.Read(buffer, 0, buffer.Length)) > 0)
       destination.Write(buffer, 0, count);
}

The WebClient class is much easier to use and I suggest using that instead.