views:

1008

answers:

2

I have a fat VB.NET Winform client that is using the an old asmx style web service. Very often, when I perform query that takes a while or pass a large amt of data to a web service in a dataset, I get the subject error.

The error seems to occur in < 1 min, which is far less than the web service timeout value that I have set or the timeout value on the ADO Command object that is performing the query within the web server.

It seems to occur whenever I am performing a large query that expects to return a lot of rows or when I am sending up a large amount of data to the web service. For example, it just occurred when I was passing a large dataset to the web server:

System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
   at System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Smit.Pipeline.Bo.localhost.WsSR.SaveOptions(String emailId, DataSet dsNeighborhood, DataSet dsOption, DataSet dsTaskApplications, DataSet dsCcUsers, DataSet dsDistinctUsers, DataSet dsReferencedApplications) in C:\My\Code\Pipeline2\Smit.Pipeline.Bo\Web References\localhost\Reference.vb:line 944
   at Smit.Pipeline.Bo.Options.Save(TaskApplications updatedTaskApplications) in 

I've been looking a tons of postings on this error and it is surprising at how varied the circumstances which cause this error are. I've tried messing with Wireshark, but I am clueless how to use it.

This application only has about 20 users at any one time and I am able to reproduce this error in the middle of the night when probably no one is using the app, so I don't think that the number of requests to the web server or to the database is high. I'm probably the only person using the app right now and I just got the error now. It seems to have to do everything with the amt of data being passed in either direction.

This error is really chronic and killing me. Please help.

+1  A: 

Check your client's app.config binding setting and see if you have specified a Max message size. The default value for this is 65536

To change this, either put it in your binding configuration (maxReceivedMessageSize, maxBufferSize and maxArrayLength are key properties for this) in your app.config, or programmatically by changing the property on the binding, like so

System.ServiceModel.BasicHttpBinding binding = new System.ServiceModel.BasicHttpBinding();

// if you are getting a LOT of data back, you will need to up Message Size
binding.MaxReceivedMessageSize = int.MaxValue; 

If this doesnt solve your issue, you will have to check the server side's event log for details. The message means that the client didnt expect the connection to close, but it did, and it could mean many different things.

If possible, use WCF for web services. It solves a lot of problems that ASMX services still suffer from.

To increase the server side transmission limit and execution timeout, use this

    <configuration>
 <system.web> 
<httpRuntime maxMessageLength="409600" executionTimeoutInSeconds="300"/> 
</system.web> </configuration> 
Lerxst
It's not the size of the msg, tho that is a good guess. I know so because I have encountered the error in debug mode and then reset the execution down the same query path again and it works fine the 2nd time. The query executed should have been the exact same query.
Velika
As I try to capture more error info, I also see this error : Unable to read data from the transport connection
Velika
Strange, but it still may have something to do with the size of the message or the execution timeout. Is it possible that the message the first time could have been larger than the message the second time? Do you get any details with the "Unable to read data" message? Check your web.config and try including the code in my edit to increase the message size to 400MB and set a 5min timeout.
Lerxst
Although I sometimes get an error and sometimes do not, I do not believe that the size of the data being returned had changed. I am 99.99% sure. I am running an app that is used by a very small group and I was testing at 3AM. The "Unable to read data" msg, I see now, was also in my orig error msg. If I read it correctly, that is not the underlying inner error that was first encountered, but instead the "forcibly closed" error was. I did make the web config changes you recommended and unfor, they did not help. Vote up of course, for useful suggestions, but still no solution. :-)
Velika
Is there anything in the server's event log? do you notice a consistent time interval between calling the service and it's failure?
Lerxst
I looked in the event logs of the web server and the database server and saw nothing related. I was surprised not to find anything. It doesn't appear to be a consistent time interval, but instead random.
Velika
we need to find something that reliably reproduces this issue. Is there any scenario where this does not fail 100%? such as a query that only returns a smaller subset of data?Your Original Post makes it seem like you think its the volume of data being passed across the wire, however if that data is not changing, then it doesnt make sense to have anything to do with that
Lerxst
Your question was "Is there any scenario where this does not fail 100%?" The answer is: It never fails 100% of the time. For operations that involve less data being passed, it works 100% of the time, for a "larger" volumes it fails occasionally, say 1 out of 3 times, though the operation appears to be identical each time it is expected. For very large amts of data, I think it would fail consistantly 100% of the time. I agree that it seems odd that if we were hitting an upper limit that it would fail 100% of the time as soon as the threshol;d was reached, yet there seems to be a clear trend.
Velika
when I say "does not fail 100% of the time" i mean is there any scenario where it works perfectly all the time, such as when the message size is cut in half, or something similar
Lerxst
Yes, when the message size is very small, it works 100% of the time.
Velika
I'm placing a call with MS. They're doing network traces. Let's see what they have to say...
Velika
A: 

I was having the exact same issue. Web service calls would fail with the same intermittent exception (~ once a day). It wasn't related to too large packet sizes or too small timeouts.

I ended up just putting in some retry logic which has worked around the problem. See: http://stackoverflow.com/questions/2270271/how-can-i-improve-this-exception-retry-scenario

Did you ever find a better solution?

robbie