[com.ctc.wstx.exc.WstxIOException: Socket closed Exception with WebLogic WebService interop with .NET WCF

Question

We have an application that accepts an uploaded file, but intermittently throws a Socket closed exception.

Client is .NET using WCF to connect to a WebService in WebLogic 10.3 using the JAX-WS annotations to create the web service.

Intermittently, while uploading large files, we receive Socket closed exceptions and are having trouble diagnosing the problem.

Application and use case info: - Files are from Excel with about 9k rows - A database insert after the upload completes typically takes 7 min. to complete. Whenever it takes longer than that, we see these errors. - Infrastructure is: .NET Client -> WebLogic 10.3 on Linux -> Oracle - We can't figure out what is throwing the error. Is it the WebLogic server stuck thread? Is it a missing annotation in the JAX-WS service? Is it Linux (the host)?

Questions we have are: - Does Linux need to be configured to allow long-running requests? - Is there a configuration for the JAX-WS to allow a long-running request?

Here is a complete stack trace:

    Root cause of ServletException.
javax.xml.ws.WebServiceException: javax.xml.bind.MarshalException
 - with linked exception:
[com.ctc.wstx.exc.WstxIOException: Socket closed]
    at com.sun.xml.ws.message.jaxb.JAXBMessage.writePayloadTo(JAXBMessage.java:322)
    at com.sun.xml.ws.message.AbstractMessageImpl.writeTo(AbstractMessageImpl.java:142)
    at com.sun.xml.ws.encoding.MtomCodec.encode(MtomCodec.java:158)
    at com.sun.xml.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:258)
    at com.sun.xml.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:320)
    at com.sun.xml.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:93)
    at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:454)
    at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:244)
    at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:134)
    at weblogic.wsee.jaxws.HttpServletAdapter$AuthorizedInvoke.run(HttpServletAdapter.java:272)
    at weblogic.wsee.jaxws.HttpServletAdapter.post(HttpServletAdapter.java:185)
    at weblogic.wsee.jaxws.JAXWSServlet.doPost(JAXWSServlet.java:180)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at weblogic.wsee.jaxws.JAXWSServlet.service(JAXWSServlet.java:64)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
    at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
    at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogicx.servlet.gzip.filter.GZIPFilter.doFilter(GZIPFilter.java:63)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3496)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic.security.service.SecurityManager.runAs(Unknown Source)
    at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
    at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
    at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
javax.xml.bind.MarshalException
 - with linked exception:
[com.ctc.wstx.exc.WstxIOException: Socket closed]
    at com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:286)
    at com.sun.xml.bind.v2.runtime.BridgeImpl.marshal(BridgeImpl.java:90)
    at com.sun.xml.bind.api.Bridge.marshal(Bridge.java:107)
    at com.sun.xml.ws.message.jaxb.JAXBMessage.writePayloadTo(JAXBMessage.java:317)
    at com.sun.xml.ws.message.AbstractMessageImpl.writeTo(AbstractMessageImpl.java:142)
    at com.sun.xml.ws.encoding.MtomCodec.encode(MtomCodec.java:158)
    at com.sun.xml.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:258)
    at com.sun.xml.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:320)
    at com.sun.xml.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:93)
    at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:454)
    at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:244)
    at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:134)
    at weblogic.wsee.jaxws.HttpServletAdapter$AuthorizedInvoke.run(HttpServletAdapter.java:272)
    at weblogic.wsee.jaxws.HttpServletAdapter.post(HttpServletAdapter.java:185)
    at weblogic.wsee.jaxws.JAXWSServlet.doPost(JAXWSServlet.java:180)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at weblogic.wsee.jaxws.JAXWSServlet.service(JAXWSServlet.java:64)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
    at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
    at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogicx.servlet.gzip.filter.GZIPFilter.doFilter(GZIPFilter.java:63)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3496)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic.security.service.SecurityManager.runAs(Unknown Source)
    at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
    at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
    at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
com.ctc.wstx.exc.WstxIOException: Socket closed
    at com.ctc.wstx.sw.BaseNsStreamWriter.doWriteStartTag(BaseNsStreamWriter.java:579)
    at com.ctc.wstx.sw.SimpleNsStreamWriter.writeStartOrEmpty(SimpleNsStreamWriter.java:273)
    at com.ctc.wstx.sw.BaseNsStreamWriter.writeStartElement(BaseNsStreamWriter.java:319)
    at com.sun.xml.ws.util.xml.XMLStreamWriterFilter.writeStartElement(XMLStreamWriterFilter.java:183)
    at com.sun.xml.bind.v2.runtime.output.XMLStreamWriterOutput.beginStartTag(XMLStreamWriterOutput.java:113)
    at com.sun.xml.bind.v2.runtime.output.XmlOutputAbstractImpl.beginStartTag(XmlOutputAbstractImpl.java:98)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.leafElement(XMLSerializer.java:319)
    at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$StringImpl.writeLeafElement(RuntimeBuiltinLeafInfoImpl.java:158)
    at com.sun.xml.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.writeLeafElement(TransducedAccessor.java:250)
    at com.sun.xml.bind.v2.runtime.property.SingleElementLeafProperty.serializeBody(SingleElementLeafProperty.java:98)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.serializeItem(ArrayElementNodeProperty.java:65)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.serializeListBody(ArrayElementProperty.java:168)
    at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.serializeBody(ArrayERProperty.java:152)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.serializeItem(ArrayElementNodeProperty.java:65)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.serializeListBody(ArrayElementProperty.java:168)
    at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.serializeBody(ArrayERProperty.java:152)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.property.SingleElementNodeProperty.serializeBody(SingleElementNodeProperty.java:150)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:277)
    at com.sun.xml.bind.v2.runtime.BridgeImpl.marshal(BridgeImpl.java:90)
    at com.sun.xml.bind.api.Bridge.marshal(Bridge.java:107)
    at com.sun.xml.ws.message.jaxb.JAXBMessage.writePayloadTo(JAXBMessage.java:317)
    at com.sun.xml.ws.message.AbstractMessageImpl.writeTo(AbstractMessageImpl.java:142)
    at com.sun.xml.ws.encoding.MtomCodec.encode(MtomCodec.java:158)
    at com.sun.xml.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:258)
    at com.sun.xml.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:320)
    at com.sun.xml.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:93)
    at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:454)
    at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:244)
    at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:134)
    at weblogic.wsee.jaxws.HttpServletAdapter$AuthorizedInvoke.run(HttpServletAdapter.java:272)
    at weblogic.wsee.jaxws.HttpServletAdapter.post(HttpServletAdapter.java:185)
    at weblogic.wsee.jaxws.JAXWSServlet.doPost(JAXWSServlet.java:180)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at weblogic.wsee.jaxws.JAXWSServlet.service(JAXWSServlet.java:64)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
    at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
    at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogicx.servlet.gzip.filter.GZIPFilter.doFilter(GZIPFilter.java:63)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3496)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic.security.service.SecurityManager.runAs(Unknown Source)
    at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
    at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
    at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
java.net.SocketException: Socket closed
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
    at weblogic.servlet.internal.ChunkOutput.writeChunkTransfer(ChunkOutput.java:525)
    at weblogic.servlet.internal.ChunkOutput.writeChunks(ChunkOutput.java:504)
    at weblogic.servlet.internal.ChunkOutput.flush(ChunkOutput.java:382)
    at weblogic.servlet.internal.ChunkOutput.checkForFlush(ChunkOutput.java:469)
    at weblogic.servlet.internal.ChunkOutput.write(ChunkOutput.java:304)
    at weblogic.servlet.internal.ChunkOutputWrapper.write(ChunkOutputWrapper.java:139)
    at weblogic.servlet.internal.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:169)
    at com.ctc.wstx.io.UTF8Writer.write(UTF8Writer.java:139)
    at com.ctc.wstx.sw.BufferingXmlWriter.flushBuffer(BufferingXmlWriter.java:1103)
    at com.ctc.wstx.sw.BufferingXmlWriter.fastWriteRaw(BufferingXmlWriter.java:1149)
    at com.ctc.wstx.sw.BufferingXmlWriter.writeStartTagStart(BufferingXmlWriter.java:697)
    at com.ctc.wstx.sw.BaseNsStreamWriter.doWriteStartTag(BaseNsStreamWriter.java:576)
    at com.ctc.wstx.sw.SimpleNsStreamWriter.writeStartOrEmpty(SimpleNsStreamWriter.java:273)
    at com.ctc.wstx.sw.BaseNsStreamWriter.writeStartElement(BaseNsStreamWriter.java:319)
    at com.sun.xml.ws.util.xml.XMLStreamWriterFilter.writeStartElement(XMLStreamWriterFilter.java:183)
    at com.sun.xml.bind.v2.runtime.output.XMLStreamWriterOutput.beginStartTag(XMLStreamWriterOutput.java:113)
    at com.sun.xml.bind.v2.runtime.output.XmlOutputAbstractImpl.beginStartTag(XmlOutputAbstractImpl.java:98)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.leafElement(XMLSerializer.java:319)
    at com.sun.xml.bind.v2.model.impl.RuntimeBuiltinLeafInfoImpl$StringImpl.writeLeafElement(RuntimeBuiltinLeafInfoImpl.java:158)
    at com.sun.xml.bind.v2.runtime.reflect.TransducedAccessor$CompositeTransducedAccessorImpl.writeLeafElement(TransducedAccessor.java:250)
    at com.sun.xml.bind.v2.runtime.property.SingleElementLeafProperty.serializeBody(SingleElementLeafProperty.java:98)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.serializeItem(ArrayElementNodeProperty.java:65)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.serializeListBody(ArrayElementProperty.java:168)
    at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.serializeBody(ArrayERProperty.java:152)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.serializeItem(ArrayElementNodeProperty.java:65)
    at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.serializeListBody(ArrayElementProperty.java:168)
    at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.serializeBody(ArrayERProperty.java:152)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.property.SingleElementNodeProperty.serializeBody(SingleElementNodeProperty.java:150)
    at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.serializeBody(ClassBeanInfoImpl.java:322)
    at com.sun.xml.bind.v2.runtime.XMLSerializer.childAsXsiType(XMLSerializer.java:681)
    at com.sun.xml.bind.v2.runtime.MarshallerImpl.write(MarshallerImpl.java:277)
    at com.sun.xml.bind.v2.runtime.BridgeImpl.marshal(BridgeImpl.java:90)
    at com.sun.xml.bind.api.Bridge.marshal(Bridge.java:107)
    at com.sun.xml.ws.message.jaxb.JAXBMessage.writePayloadTo(JAXBMessage.java:317)
    at com.sun.xml.ws.message.AbstractMessageImpl.writeTo(AbstractMessageImpl.java:142)
    at com.sun.xml.ws.encoding.MtomCodec.encode(MtomCodec.java:158)
    at com.sun.xml.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:258)
    at com.sun.xml.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:320)
    at com.sun.xml.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:93)
    at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:454)
    at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:244)
    at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:134)
    at weblogic.wsee.jaxws.HttpServletAdapter$AuthorizedInvoke.run(HttpServletAdapter.java:272)
    at weblogic.wsee.jaxws.HttpServletAdapter.post(HttpServletAdapter.java:185)
    at weblogic.wsee.jaxws.JAXWSServlet.doPost(JAXWSServlet.java:180)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at weblogic.wsee.jaxws.JAXWSServlet.service(JAXWSServlet.java:64)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
    at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
    at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
    at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogicx.servlet.gzip.filter.GZIPFilter.doFilter(GZIPFilter.java:63)
    at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
    at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3496)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
    at weblogic.security.service.SecurityManager.runAs(Unknown Source)
    at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
    at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
    at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

Answer 1

If it is always happening after the upload takes a specific amount of time then it is probably:

• A timeout
• A problem with the size of the document

The relevant default timeout in WCF is 10 mins, so the best bet is that it has someting to do with the size. (unless the default is different in web logic / your application)

Try increasing the MaxReceivedMessageSize in the WCF configuration file.