Unhandled exception while downloading web page using HttpWebRequest and StreamReader

Question

This is a C# app using .NET 4.0.

Background:

I have been trying to refine a portion of an app which downloads files from multiple web servers. Everything is working pretty well except for an occasional situation wherein the remote computer closes the connection. I assume this may be due to a variety of factors, including network problems, power problems, etc. I'd like to simply skip the download in the event that the connection closes. However at present the application causes an exception which is picked up by the handler for AppDomain.CurrentDomain.UnhandledException. The stack trace is as follows:

Stack Trace:

Unable to read data from the transport connection:
An existing connection was forcibly closed by the remote host.
at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.StreamReader.ReadBuffer() at System.IO.StreamReader.ReadToEnd()
at ProjectName.ClassNetwork.DownloadFile(String _IP, Int32 _Port, String _File)
at ProjectName.FormMain.GetIFile(ClassSensor Sensor, String& RawHolder, String[] FileData)
at ProjectName.FormMain.GetLegacyData(ClassSensor Sensor)
at ProjectName.FormMain.threader_Download(Object SensorObject)
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart(Object obj)

I originally had the try/catch for the WebException and Exception but they didn't catch this particular error. I added the innermost try/catch in an attempt to avoid the problem, but to no avail. I should mention that this method is called by threads other than the UI thread, which may be partly why the try/catch blocks are not seeming to work. Forgive me for being a newbie when it comes to multi-threaded error catching!

Question:

• How can I improve this method and correctly handle situations where the connection fails midstream?

---

Code:

private static object[] DownloadFile(string _IP, int _Port, string _File)
{
    string uri = String.Format("http://{0}:{1}/{2}", _IP, _Port, _File);
    string Status = String.Empty;
    string Data = String.Empty;
    HttpWebResponse Response = null;

    try
    {
        HttpWebRequest webReq = (HttpWebRequest) WebRequest.Create(uri);
        webReq.Timeout = 10000;
        webReq.ReadWriteTimeout = 30000;
        Response = (HttpWebResponse) webReq.GetResponse();
        using (Stream dataStream = Response.GetResponseStream())
            if (dataStream != null)
                using (StreamReader reader = new StreamReader(dataStream))
                    try
                    {
                        // This line causes crashes if remote computer closes connection
                        Data = reader.ReadToEnd();
                    }
                    catch { } // Inner try/catch added to no avail
        Response.Close();
    }
    catch (WebException exc)
    {
        // Connection Error
        Status = exc.Status.ToString();
        Data = String.Empty;
    }
    catch (Exception exc)
    {
        // Other error
        Status = exc.Message;
        Data = String.Empty;
    }

    if (Response != null && Response.StatusCode != HttpStatusCode.OK)
    {
        Status = Response.StatusCode.ToString();
        Data = String.Empty;
    }

    return new object[] { Status, Data };
}

Trace Log:

Per Feroze's suggestion, I ran a trace point and this is the result:

System.Net Error: 0 : [2164] Exception in the
#46104728::UnhandledExceptionHandler - Stream was not readable.
System.Net Error: 0 : [2164]     at System.IO.BinaryReader..ctor(Stream input, Encoding encoding)
at ProjectName.ClassNetwork.DLStream(ClassSensor Sensor)
at ProjectName.ClassNetwork.DownloadFile(ClassSensor Sensor)
at ProjectName.FormMain.GetStreamFile(ClassSensor Sensor)
at ProjectName.FormMain.threader_Download(Object SensorObject)
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncctx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart(Object obj)

Answer 1

From the stack, I do not see how this is an exception thrown by a threadpool thread that you cannot catch. Obviously from the stack it seems as if it was thrown by the ConnectStream::Read method which was called on your application thread.

That said, you should be prepared to catch and retry any exceptions that are retryable. For eg, this one (connection forcibly closed by remote host) could be retryable provided you first figure out the root cause of the failure. For that, it might be best to create a system.net tracelog for your application and post the logfile on pastebin.com, and put a link to that here.