I have an old ASP.NET application. We send out httpWebRequest to a remote REST server and retrieve XML back, most of the time the application works fine. Recently, we got some high CPU usage issue several times a day.
During the high CPU usage time, we monitored those httpWebRequest connections (by checking netstat for the w3wp process). At the very beginning, the connections change to "CLOSE_WAIT" status from "ESTABLISHED", then after those connections timeout, those connections disappeared one by one, and then there is no connection any more.
After resetting the IIS, when the w3wp.exe process start again, we still could not find any connections to httpWebRequest target server. So the CPU usage keep staying at high level. Even after several round of reset, it won't solve the issue, until we saw some connections start to connect to httpWebRequest target server, the CPU usage went down.
I actually thought it could be the issue of my code not handling the httpWebRequest properly, I posted another question here: How to close underlying connections after catch httpwebrequest timeout?.
As mentioned in that question, I also found lots of timeout exceptions for System.Net.HttpWebRequest.GetResponse()
. We found 3500 of the same exception within 5 minutes when CPU usage is really high.
What could cuase this type of issue and what could be the medicine? Why won't the application send out request any more (since there is no connection in netstat)?
Here is the source code just in case:
System.Net.HttpWebResponse httpWebResponse = null;
System.IO.Stream stream = null;
XmlTextReader xmlTextReader = null;
try
{
System.Net.HttpWebRequest httpWebRequest = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(request);
httpWebRequest.ReadWriteTimeout = 10000;
httpWebRequest.Timeout = 10000;
httpWebRequest.KeepAlive = false;
httpWebRequest.Method = "GET";
httpWebResponse = (System.Net.HttpWebResponse)httpWebRequest.GetResponse();
stream = httpWebResponse.GetResponseStream();
xmlTextReader = new XmlTextReader(stream);
xmlTextReader.Read();
xmlDocument.Load(xmlTextReader);
//Document processing code.
//...
}
catch
{
//Catch blcok with error handle
}
finally
{
if (xmlTextReader != null)
xmlTextReader.Close();
if (httpWebResponse != null)
httpWebResponse.Close();
if (stream != null)
stream.Close();
}