What's causing my java.net.SocketException: Connection reset?

Question

We are seeing frequent java.net.SocketException: Connection reset errors in our logs for a component that calls a third party Web service that sends SMS messages.

Our application is written in Java and is sat on top of Tomcat 5.5. It was written by contractors who are no longer with us. The current team has no real Java expertise, and we're unsure as to where the Connection reset error is actually coming from, and how to go about debugging.

The issue appears to be completely intermittent, and unrelated to the messages we are attempting to send.

Any suggestions on what the typical causes of this exception might be, and how we might proceed, are welcomed.

The whole call stack is included below for completeness.

(com.companyname.mtix.sms is our component)

java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:168)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
    at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:77)
    at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
    at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1115)
    at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1832)
    at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590)
    at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
    at com.companyname.mtix.sms.services.impl.message.SendTextMessage.sendTextMessage(SendTextMessage.java:127)
    at com.companyname.mtix.sms.services.MessageServiceImpl.sendTextMessage(MessageServiceImpl.java:125)
    at com.companyname.mtix.sms.services.remote.MessageServiceRemoteImpl.sendTextMessage(MessageServiceRemoteImpl.java:43)
    at sun.reflect.GeneratedMethodAccessor203.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
    at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
    at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
    at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
    at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
    at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
    at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:453)
    at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
    at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    at com.companyname.mtix.sms.http.filters.NoCacheFilter.doFilter(NoCacheFilter.java:63)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    at com.companyname.mtix.sms.http.filters.MessageFilter.doFilter(MessageFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:61)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    at org.ajaxanywhere.AAFilter.doFilter(AAFilter.java:46)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
    at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    at java.lang.Thread.run(Thread.java:595)

Edit

The line of our code on which the exception gets thrown from is the last line in the code snippet below

String aggregatorResponse = null;
HttpClient httpClient = prepareHttpClient( username, password );
PostMethod postMethod = preparePostMethod( textUrl );

try {
  SybaseTextMessageBuilder builder = new SybaseTextMessageBuilder();
  URL notifyUrl = buildNotificationUrl( textMessage, codeSetManager );
  String smsRequestDocument = builder.buildTextMessage( textMessage, notifyUrl );
  LOG.debug( "Sybase MT document created as: \n" + smsRequestDocument );

  postMethod.setRequestEntity( new StringRequestEntity( smsRequestDocument ) );
  LOG.debug( "commiting SMS to aggregator: " + textMessage.toString() );
  int httpStatus = httpClient.executeMethod( postMethod );

Answer 1

The Exception means that the socket was closed unexpectedly from the other side. Since you are calling a web service, this should not happen - most likely you're sending a request that triggers a bug in the web service.

Try logging the entire request in those cases, and see if you notice anything unusual. Otherwise, get in contact with the web service provider and send them your logged problematical request.

Answer 2

The javadoc for SocketException states that it is

Thrown to indicate that there is an error in the underlying protocol such as a TCP error

In your case it seems that the connection has been closed by the server end of the connection. This could be an issue with the request you are sending or an issue at their end.

To aid debugging you could look at using a tool such as Wireshark to view the actual network packets. Also, is there an alternative client to your Java code that you could use to test the web service? If this was successful it could indicate a bug in the Java code.

As you are using Commons HTTP Client have a look at the Common HTTP Client Logging Guide. This will tell you how to log the request at the HTTP level.

Answer 3

This error occurs on the server side when the client closed the socket connection before the response could be returned over the socket. In a web app scenario not all of these are dangerous, since they can be created manually. For example, by quitting the browser before the reponse was retrieved.

Answer 4

This error happens on your side, NOT server side. If on server side, it should say "java.net.SocketException reset by peer". The cause is the connection inside HttpClient is staled. Check staled connection for SSL does not fix this error. Solution: dump your client and recreate.

Mike Huang.

Answer 5

I get this error all the time and consider it normal. It happens when one side tries to read when the other side has already hung up. Thus depending on the protocol this may or may not designate a problem. If my client code specifically indicates to the server that it is going to hang up, then both client and server can hang up at the same time and this message would not happen. The way I implement my code is for the client to just hang up without saying good buy. The server can then catch the error and ignore it. In the context of http, I believe one level of the protocol allows more then one request per connection while the other doesn't. Thus you can see how potentially one side could keep hanging up on the other. I doubt the error you are receiving is of any piratical concern and you could simply catch it to keep it from filling up your log files.