What's causing my java.net.SocketException: Connection reset? [duplicate]

Question

We are seeing frequent but intermittent java.net.SocketException: Connection reset errors in our logs. We are unsure as to where the Connection reset error is actually coming from, and how to go about debugging.

The issue appears to be unrelated to the messages we are attempting to send. Note that the message is not connection reset by peer.

Any suggestions on what the typical causes of this exception might be, and how we might proceed?

Here is a representative stack trace (com.companyname.mtix.sms is our component):

    java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:168)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
        at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:77)
        at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
        at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1115)
        at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1832)
        at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590)
        at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995)
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
        at com.companyname.mtix.sms.services.impl.message.SendTextMessage.sendTextMessage(SendTextMessage.java:127)
        at com.companyname.mtix.sms.services.MessageServiceImpl.sendTextMessage(MessageServiceImpl.java:125)
        at com.companyname.mtix.sms.services.remote.MessageServiceRemoteImpl.sendTextMessage(MessageServiceRemoteImpl.java:43)
        at sun.reflect.GeneratedMethodAccessor203.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
        at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
        at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
        at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
        at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
        at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
        at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:453)
        at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
        at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
        at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at com.companyname.mtix.sms.http.filters.NoCacheFilter.doFilter(NoCacheFilter.java:63)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at com.companyname.mtix.sms.http.filters.MessageFilter.doFilter(MessageFilter.java:53)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:61)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:77)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.ajaxanywhere.AAFilter.doFilter(AAFilter.java:46)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
        at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
        at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
        at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
        at java.lang.Thread.run(Thread.java:595)
    

---

Our component is a web application, running under Tomcat, that calls a third party Web service that sends SMS messages, it so happens. The line of our code on which the exception gets thrown from is the last line in the code snippet below.

String aggregatorResponse = null;
HttpClient httpClient = prepareHttpClient( username, password );
PostMethod postMethod = preparePostMethod( textUrl );

try {
  SybaseTextMessageBuilder builder = new SybaseTextMessageBuilder();
  URL notifyUrl = buildNotificationUrl( textMessage, codeSetManager );
  String smsRequestDocument = builder.buildTextMessage( textMessage, notifyUrl );
  LOG.debug( "Sybase MT document created as: \n" + smsRequestDocument );

  postMethod.setRequestEntity( new StringRequestEntity( smsRequestDocument ) );
  LOG.debug( "commiting SMS to aggregator: " + textMessage.toString() );
  int httpStatus = httpClient.executeMethod( postMethod );

Peccadillo

As you see from the full stack it starts with read operation . Your client is gone but you try to read from its stream. See my ServerSocket solution https://mcmap.net/q/135407/-java-net-socketexception-connection-reset. – Lyly

For people encountered this issue when using Go, github.com/golang/go/issues/50984 this issue will cause Connection reset because the extraneous TLS connection closed in the handshake process after http.Do done – Leong

Answer 1

The javadoc for SocketException states that it is

Thrown to indicate that there is an error in the underlying protocol such as a TCP error

In your case it seems that the connection has been closed by the server end of the connection. This could be an issue with the request you are sending or an issue at their end.

To aid debugging you could look at using a tool such as Wireshark to view the actual network packets. Also, is there an alternative client to your Java code that you could use to test the web service? If this was successful it could indicate a bug in the Java code.

As you are using Commons HTTP Client have a look at the Common HTTP Client Logging Guide. This will tell you how to log the request at the HTTP level.

Vegetal

I don't believe that Connection Reset means that the server closed the connection (via sending a FYN flag). I believe that it means it is no longer responding to ACK flags, or that it is not responding per TCP/IP). If the FYN flag was sent, or rather the server closed the connection, you would just get -1 from read(), and an EOFException. When I get home I plan on confirming this either way. – Milamilady

In response to Zombies comment, did anyone ever find out if the connection reset means the server has closed the connection? – Corded

In my experience with a similar problem in the past, the cause was a bad network connection on the server side that causes too many connections timeout. The problem with the timeout of a Socket is that it's default behavior is not to timeout (infinite timeout). (Continues) – Scholasticate

(Continuation) Even HTTPClient doesn't set a default timeout on the created sockets. On the other hand, the server side all sockets must timeout after a few minutes or the connections will get stuck (very bad idea on a server). So if the connection times out on the server side, the client will get a connection reset (server closed the connection), if the client times out first the exception will be a ConnectionTimeoutException. Since this is a transient problem, the solution is to catch and log all this exceptions and retry again. – Scholasticate

@FilipePalrinhas you're very right about the fact that HttpClient doesn't set a finite timeout on the socket connection. – Centre

@Corded There are several possible causes. See my answer in the duplicate. – Branch

@Milamilady It means the server has sent RST. BTW the other flag you mention is spelt FIN, not FYN. – Branch

Answer 2

This error happens on your side and NOT the other side. If the other side reset the connection, then the exception message should say:

java.net.SocketException reset by peer

The cause is the connection inside HttpClient is stale. Check stale connection for SSL does not fix this error. Solution: dump your client and recreate.

Awed

Can you elaborate more please.. this is not very clear.. What do you mean by "dump" your client? – Predatory

Answer 3

If you experience this trying to access Web services deployed on a Glassfish3 server, you might want to tune your http-thread-pool settings. That fixed SocketExceptions we had when many concurrent threads was calling the web service.

1. Go to admin console
2. Navigate to "Configurations"->"Server config"->"Thread pools"->"http-thread-pool".
3. Change setting "Max Thread Pool Size" from 5 to 32
4. Change setting "Min Thread Pool Size" from 2 to 16
5. Restart Glassfish.

Demott

This helped me on Glassfish 4 also – Rimrock

Answer 4

I did also stumble upon this error. In my case the problem was I was using JRE6, with support for TLS1.0. The server only supported TLS1.2, so this error was thrown.

Peppergrass

Very difficult to believe. You should have got an SSLHandshakeException. – Branch

@EJP: My answer was written a long time ago, so I can't check out the issue again. I only remember stumbling to this error + updaten to TLS 1.2 fixed the problem. After a short googeling, I see others got the java.net.SocketException as well, when using TLS1.0 in combination of a server supporting TLS1.2 at mimimum. For example assuresign.tenderapp.com/kb/how-to/…. – Peppergrass

Yes, this can happen with Java 7 too. The default TLS implementation is TLSv1, which is sent in a ClientHello; if the server doesn't support it, you'll get a Connection reset, not a SSLHandshakeException. – Snuffy

Answer 5

In my case, this was because my Tomcat was set with an insufficient maxHttpHeaderSize for a particularly complicated SOLR query.

Hope this helps someone out there!

Racy

Answer 6

This error occurs on the server side when the client closed the socket connection before the response could be returned over the socket. In a web app scenario not all of these are dangerous, since they can be created manually. For example, by quitting the browser before the reponse was retrieved.

Ulent

While true, this question is about a client-side component where you would expect the server to finish the exchange. – Selfabuse

Answer 7

I get this error all the time and consider it normal.

It happens when one side tries to read when the other side has already hung up. Thus depending on the protocol this may or may not designate a problem. If my client code specifically indicates to the server that it is going to hang up, then both client and server can hang up at the same time and this message would not happen.

The way I implement my code is for the client to just hang up without saying goodbye. The server can then catch the error and ignore it. In the context of HTTP, I believe one level of the protocol allows more then one request per connection while the other doesn't.

Thus you can see how potentially one side could keep hanging up on the other. I doubt the error you are receiving is of any piratical concern and you could simply catch it to keep it from filling up your log files.

Reenter

The protocol is clearly HTTP in this case, and so it should not happen, except for the rare cases that the other end of the connection exits. – Caseate

No. The sender can close while there are still many kilobytes of data left unread by the receiver. In this circumstance the receiver will read all the data and then get an end of stream, not a reset. – Branch

Answer 8

This is an old thread, but I ran into java.net.SocketException: Connection reset yesterday.

The server-side application had its throttling settings changed to allow only 1 connection at a time! Thus, sometimes calls went through and sometimes not. I solved the problem by changing the throttling settings.

Thorma

Answer 9

The Exception means that the socket was closed unexpectedly from the other side. Since you are calling a web service, this should not happen - most likely you're sending a request that triggers a bug in the web service.

Try logging the entire request in those cases, and see if you notice anything unusual. Otherwise, get in contact with the web service provider and send them your logged problematical request.

Garton

No it doesn't. That would be 'connection reset by peer'. – Branch

Answer 10

I know this thread is little old, but would like to add my 2 cents. We had the same "connection reset" error right after our one of the releases.

The root cause was, our apache server was brought down for deployment. All our third party traffic goes thru apache and we were getting connection reset error because of it being down.

Diarmuid

Answer 11

I got this error when the text file I was trying to read contained a string that matched an antivirus signature on our firewall.

Kean

Answer 12

FWIW, I was getting this error when I was accidentally making a GET request to an endpoint that was expecting a POST request. Presumably that was just that particular servers way of handling the problem.

Tutuila

Answer 13

I was getting exactly that error too: Connection reset by peer. The exception was being raised by Spring's REST template upon running the postForObject() method. For me the problem was too long HTTP URL request. So first check whether the URL produced is what it should be and, if your server really should be able to handle requests of that length, simply go to server's configuration and raise the default allowed length of URL requests.

That solved the problem for me, but be aware: the application might not run on some internet browsers, especially old ones, as they have fixed max length of URL requests.

Hope it helps...

Dollfuss

Connection reset is not the same as Connection reset by peer. – Pioneer

Answer 14

I was getting this error because the port I tried to connect to was closed.

Beneficent

That would have caused a connect timeout or connection refusal. – Branch